Apparatus and computer implemented method in marine vessel data system for generating anomaly heatmap information using neural network

ABSTRACT

A computer implemented method and apparatus for a marine vessel data system, the method comprising: receiving data from at least one sensor configured to measure vibration and operationally arranged to the marine vessel to provide time-domain reference sensor data; maintaining the time-domain reference sensor data within a data storage system; generating a Fast Fourier Transform (FFT) on the time-domain reference sensor data to provide a plurality of reference spectra files in frequency-domain, wherein each reference spectra file comprises spectra data defined by amplitude information and frequency information, and each spectra file is associated with condition information determined based on collection of the time-domain reference sensor data; normalizing each reference spectra file by converting the frequency information to order information using the condition information to provide normalized reference spectra files; and training a convolutional autoencoder type of neural network using the normalized reference spectra files.

TECHNICAL FIELD

The present application generally relates to an apparatus, method andsoftware code for analysing vibration data for detecting anomalities inmarine vessel data system.

BRIEF DESCRIPTION OF RELATED DEVELOPMENTS

This section illustrates useful background information without admissionof any technique described herein representative of the state of theart.

Modern vehicles, such as marine vessels, face different kinds ofefficiency and performance requirements that are becoming increasinglymore stringent. New legislation in the United States and European Unionwithin the past few years has set, for example, fuel economy andemissions targets not readily achievable with prior known vehicle andfuel technology.

To address these increasing standards, marine vessel equipmentmanufacturers are demanding better performance.

Anomaly detection is an important topic that has been studied withindiverse research areas and application domains. It refers to the problemof finding patterns in data that do not conform to expected behaviour.In Data Mining (DM), anomaly detection refers to the identification ofrare items and observations, which raise suspicion by showing asignificant difference from the majority of the data. The anomaliestranslate to some kind of problem such as errors in media, weatherpredictions, broken devices etc. Detecting anomalies proves effective ina wide variety of applications such as fraud detection for credit cards,insurance, intrusion detection for cyber-security, military surveillanceof enemy activities and machinery maintenance.

This topic is often referred to as condition monitoring. Any machine,whether it is a rotating equipment (a pump, a compressor, a turbine,etc.) or a non-rotating (a distillation column, a valve, etc.) willeventually reach a condition of poor health. That condition does notnecessarily have to be an actual failure or shutdown, but a condition inwhich the equipment is no longer performing at its optimal state. Thisindicates that there is a need for maintenance activity, to restore thefull operating potential of the equipment.

The domain of condition monitoring primarily refers to identifying thehealth status of the equipment and its performance status. A common wayto perform condition monitoring is to observe the installed sensormeasurements, from the machine, and to impose limits on it. If thecurrent values are within the normal bounds it is considered healthy.However, if the current values are outside the bounds, then the machineis deemed unhealthy and an alarm is sent to the monitoring system. Theexperts investigate the alarm and proceed with the necessary actions torepair the faulty machinery. However, methods like this have been provento be rather inflexible since most machines need different bounds aftersignificant time of usage. Engines, motors, propellers and similarinstruments experience different amounts of operation, ensuing multipleyears of deployment. It is common for engine experts to take intoconsideration the amount of usage each part has, in order to accuratelyevaluate the condition the machine is in. Evidently, having constantlimits for alarms is not a dynamic solution for the problem of anomalydetection.

It is known from the prior art, for example, to provide an arrangementfor taking a signal sample e.g. from an internal combustion engine andset threshold limits for pre-defined signal samples for triggeringanomalities.

However, receiving a vast number of signals from a single system with aplurality of operationally interconnected elements, and settingthresholds or triggers for certain data samples during system operationis unreliable, cumbersome, difficult and requires more complexity. Dueto even more stringent demands on efficiency and operationalreliability, a need has emerged to improve the possibilities of trackingthe operation of a marine vessel system by processing gathered data todetect anomalities.

Thus, an easy to set-up, accurate, and highly functional and reliablesolution is needed to provide more accurate system for detectinganomalities in marine vessel environment, especially for propulsionsystems.

SUMMARY

According to a first example aspect of the disclosed embodiments thereis provided a computer implemented method in a marine vessel datasystem, the method comprising:

receiving time-domain data from a sensor configured to measure vibrationof a system comprising a plurality of operationally connected parts, andoperationally arranged to the marine vessel;

generating a Fast Fourier Transform (FFT) on the time-domain data toprovide a plurality of spectra files in frequency-domain, wherein eachspectra file comprises spectra data defined by amplitude information andfrequency information, and each spectra file is associated withcondition information determined based on collection of the time-domaindata;

normalizing each spectra file by converting the frequency information toorder information using the condition information to provide normalizedspectra files;

generating actual sensor data based on the normalized spectra files;

generating predicted data for the at least one sensor as output by aconvolutional autoencoder type of neural network, wherein theconvolutional autoencoder type of neural network is configured to betrained using normalized reference spectra files, wherein eachnormalized reference spectra file comprises spectra data defined byamplitude information and order information, and each spectra file isassociated with timestamp information determined based on collection ofthe reference sensor data;

combining the predicted data with the actual sensor data for the sensorto provide error data; and

generating 3D anomaly heatmap information, wherein a first dimension isdefined by the timestamp information, a second dimension is defined bythe order information and a third dimension is defined by anomalyinformation determined based on the error data.

In an embodiment, the method further comprises:

detecting consecutive spectra files of the actual sensor data in thefirst dimension with anomaly information in the third dimension, whereinthe anomaly information exceeds pre-defined anomaly threshold;

determining at least one detected spectra file to identify orderinformation for the anomaly information;

comparing the identified order information with part frequencies of theplurality of operationally connected parts; and

determining faulty part based on the comparison.

In an embodiment, the method further comprises:

identifying order information with at least one harmonic for the anomalyinformation when determining the at least one detected spectra file;

comparing the identified order information with the at least oneharmonic and the part frequencies of the plurality of operationallyconnected parts; and

determining faulty part based on the comparison.

In an embodiment, the method further comprises:

detecting anomalies in a sub-range of the second dimension based on the3D anomaly heatmap information; and

generating second 3D anomaly heatmap information, wherein a firstdimension is defined by the timestamp information, a second dimension isdefined by the order information of the sub-range and a third dimensionis defined by anomaly information determined based on the error data.

In an embodiment, the method further comprises:

determining false anomalies based on the second 3D anomaly heatmapinformation by identifying order pole pass frequencies.

In an embodiment, the method further comprises:

detecting at least one spectra file of the actual sensor data withanomaly information in the third dimension, wherein the anomalyinformation exceeds pre-defined anomaly threshold;

determining order information for the anomaly information in thesub-range;

comparing the determined order information with order pole passfrequencies and part frequencies of the plurality of operationallyconnected parts; and

determining faulty part based on the comparison.

In an embodiment, the method further comprises:

ignoring order information relating to pole pass frequencies whendetermining faulty part.

In an embodiment, the at least one sensor comprises an accelerometer.

In an embodiment, the time-domain data is maintained in a data storagesystem.

In an embodiment, generating the predicted data comprises reconstructingdata of the at least one sensor, by the neural network.

In an embodiment, generating the predicted data comprises determiningcorrelation, by the neural network, for the at least one sensor.

In an embodiment, the neural network is trained by means of signals fromat least one sensor for determining internal neural network parameters.

In an embodiment, the neural network is used for determination of theanomaly based on the error data.

In an embodiment, the sensor is configured to measure vibration of apropulsion system comprising at least a shaft, a bearing, a propellerbase and a propeller blade, and at least one of them having pre-definedpart frequency provided for determining faulty part.

In an embodiment, the training is configured to utilize a trainingfunction that comprises at least one of the following:

Gradient descent function;

Newton's method function;

Conjugate gradient function;

Quasi-Newton method function; and

Levenberg-Marquardt function.

In an embodiment, the method further comprises:

receiving remote normalized reference spectra files originating from aremote marine vessel apparatus; and

training the convolutional autoencoder type of neural network using theremote normalized reference spectra files.

In an embodiment, the method further comprises:

receiving time-domain data from a plurality of sensors configured tomeasure vibration of a system comprising a plurality of operationallyconnected parts, and operationally arranged to the marine vessel.

In an embodiment, the method further comprises:

generating predicted data for at least one sensor as output by aconvolutional autoencoder type of neural network, wherein theconvolutional autoencoder type of neural network is configured to betrained using normalized reference spectra files, wherein eachnormalized reference spectra file comprises spectra data defined byamplitude information and order information, and each spectra file isassociated with timestamp information determined based on collection ofthe reference sensor data;

combining the predicted data with the actual sensor data for the sensorto provide error data; and

generating 3D anomaly heatmap information, wherein a first dimension isdefined by the timestamp information, a second dimension is defined bythe order information and a third dimension is defined by anomalyinformation determined based on the error data.

According to a second example aspect of the disclosed embodiments thereis provided a server apparatus in a marine vessel data system,comprising:

a communication interface;

at least one processor, and

at least one memory including computer program code;

the at least one memory and the computer program code configured to,with the at least one processor, cause the apparatus to:

-   -   receive time-domain data from a sensor configured to measure        vibration of a system comprising a plurality of operationally        connected parts, and operationally arranged to the marine        vessel;    -   generate a Fast Fourier Transform (FFT) on the time-domain data        to provide a plurality of spectra files in frequency-domain,        wherein each spectra file comprises spectra data defined by        amplitude information and frequency information, and each        spectra file is associated with condition information determined        based on collection of the time-domain data;    -   normalize each spectra file by converting the frequency        information to order information using the condition information        to provide normalized spectra files;    -   generate actual sensor data based on the normalized spectra        files;    -   generate predicted data for the at least one sensor as output by        a convolutional autoencoder type of neural network, wherein the        convolutional autoencoder type of neural network is configured        to be trained using normalized reference spectra files, wherein        each normalized reference spectra file comprises spectra data        defined by amplitude information and order information, and each        spectra file is associated with timestamp information determined        based on collection of the reference sensor data;    -   combine the predicted data with the actual sensor data for the        sensor to provide error data; and    -   generate 3D anomaly heatmap information, wherein a first        dimension is defined by the timestamp information, a second        dimension is defined by the order information and a third        dimension is defined by anomaly information determined based on        the error data.

According to a third example aspect of the disclosed embodiments thereis provided a computer program embodied on a computer readable mediumcomprising computer executable program code, which code, when executedby at least one processor of an apparatus, causes the apparatus to:

-   -   receive time-domain data from a sensor configured to measure        vibration of a system comprising a plurality of operationally        connected parts, and operationally arranged to the marine        vessel;    -   generate a Fast Fourier Transform (FFT) on the time-domain data        to provide a plurality of spectra files in frequency-domain,        wherein each spectra file comprises spectra data defined by        amplitude information and frequency information, and each        spectra file is associated with condition information determined        based on collection of the time-domain data;    -   normalize each spectra file by converting the frequency        information to order information using the condition information        to provide normalized spectra files;    -   generate actual sensor data based on the normalized spectra        files;    -   generate predicted data for the at least one sensor as output by        a convolutional autoencoder type of neural network, wherein the        convolutional autoencoder type of neural network is configured        to be trained using normalized reference spectra files, wherein        each normalized reference spectra file comprises spectra data        defined by amplitude information and order information, and each        spectra file is associated with timestamp information determined        based on collection of the reference sensor data;    -   combine the predicted data with the actual sensor data for the        sensor to provide error data; and    -   generate 3D anomaly heatmap information, wherein a first        dimension is defined by the timestamp information, a second        dimension is defined by the order information and a third        dimension is defined by anomaly information determined based on        the error data.

Different non-binding example aspects and embodiments of the disclosurehave been illustrated in the foregoing. The above embodiments are usedmerely to explain selected aspects or steps that may be utilized inimplementations of the present invention. Some embodiments may bepresented only with reference to certain example aspects of theinvention. It should be appreciated that corresponding embodiments mayapply to other example aspects as well.

BRIEF DESCRIPTION OF THE DRAWINGS

The aspects of the disclosed embodiments will be described, by way ofexample only, with reference to the accompanying drawings, in which:

FIG. 1 shows a schematic picture of a marine vessel and a marine vesseldata system according to an example embodiment;

FIG. 2 presents an example block diagram of a control apparatus within amarine vessel, for example, in which various embodiments of theinvention may be applied;

FIG. 3 presents an example block diagram of a sensor device in whichvarious embodiments of the invention may be applied;

FIG. 4 shows a schematic picture of a computer implemented method andsystem for training a neural network based ADM model according to anexample embodiment;

FIG. 5 shows a schematic picture of a computer implemented method andsystem for generating anomaly information using a neural network basedADM model according to an example embodiment;

FIG. 6 shows a schematic picture of an autoencoder architecture of aneural network configured to be trained using normalized referencespectra files according to an example embodiment;

FIG. 7 shows a schematic picture of high level steps of the proposedapproach according to an example embodiment;

FIG. 8 shows a schematic picture of using FFT for vibration analysisaccording to an example embodiment;

FIG. 9a shows a schematic picture of having three signals producing thetimeseries according to an example embodiment;

FIG. 9b shows a schematic picture of having three signals producing thefrequency series after applying FFT according to an example embodiment;

FIG. 10a shows a schematic picture of one of the spectra files plottedwith respect to the corresponding frequency according to an exampleembodiment;

FIG. 10b shows a schematic picture of one of a spectra files after ordernormalization has been applied according to an example embodiment;

FIG. 10c shows a schematic picture of one of a spectra files for thefirst 30 orders according to an example embodiment;

FIG. 11 shows a schematic picture of max pooling according to an exampleembodiment;

FIG. 12 shows a schematic picture of an example of a spectra filewithout any anomalies detected according to an example embodiment;

FIG. 13 shows a schematic picture of 3D heatmap according to an exampleembodiment;

FIG. 14 shows another 3D heatmap information according to an exampleembodiment;

FIG. 15 shows second 3D heatmap information according to an exampleembodiment;

FIG. 16 shows a schematic diagram of a sensor data item in accordancewith an example embodiment;

FIG. 17 shows a schematic picture of a system according to an exampleembodiment;

FIG. 18 presents an example block diagram of a server apparatus in whichvarious embodiments of the invention may be applied; and

FIG. 19 shows a flow diagram showing operations in accordance with anexample embodiment of the invention.

DETAILED DESCRIPTION

In the following description, like numbers denote like elements.

FIG. 1 shows a schematic picture of a marine vessel 105 and a marinevessel data system 110 according to an example embodiment.

The marine vessel data system 110 comprises a control apparatus 120configured to provide and operate an anomaly detection model (ADM) 121or at least part of the operations needed for the anomaly detectionmodel (ADM) 121, such as providing sensor data from the marine vesseldata system 100 to a remote server apparatus where the anomaly detectionmodel (ADM) is configured to be maintained and operated.

When operating a marine vessel with a plurality of sensors operationallyconnected to the marine vessel data system 110, data 122 is generated.Data 122 may be received from the plurality of sensors operationallyarranged to the marine vessel 105 to provide sensor data 124. The sensordata 124 may be maintained within a data storage system. The data 122may also comprise data from various data sources within the marinevessel data system 110, such as computer systems or communicationsystems, for example.

The dynamic anomaly detection model (ADM) 121 may be maintained andoperated by the control apparatus 120 and receives data 122 as input.Further inputs may be provided from various data sources, internal orexternal, as well as operational or environmental, as discussedthroughout the description. Alternatively, the dynamic anomaly detectionmodel (ADM) may be maintained and operated by a remote server apparatusand receive input data 122 from the control apparatus 120. In suchscenario the ADM 121 illustrated in FIG. 1 may be the plain datacollection point or collection unit to provide the data to the remoteserver apparatus for the actual anomaly detection model (ADM).

In an embodiment, the anomaly detection model 121 utilizes a neuralnetwork to predict signal behavior of equipment within the marine vessel105 with the goal of detecting abnormal behavior within the marinevessel data system 110.

In an embodiment, by establishing an extended interface between the ADMmodel 121 (or data collection unit in case the ADM model is operated atremote server apparatus) and other systems like the navigation system130, automation system 190, power generation system 140, propulsionsystem 150, fuel system 160, energy load system 170 and auxiliary sensorsystem 180, for example, it is possible to gather vast amount of complexinput data for ADM 121 to process, learn, predict and detectanomalities.

The power generation system 140 may comprise different power generationsources, such as engines operating with diesel oil, LNG, LPG, syntheticfuels or like. Propulsion system 150 may be configured to change thepropulsion energy source (e.g. electric motor powering the propulsionwherein the energy source for the energy motor is changed), or foractivating exhaust gas cleaning system (e.g. scrubber and/or SCR), forexample. By establishing an ADM 121 (or data collection unit or pointfor the remote ADM) for communicating between systems 120-190 it ispossible to use artificial neural networks as computing systems. Theneural network within the ADM 121 itself may not be a single algorithmbut a framework for many different machine learning algorithms to worktogether and process complex data inputs. Such systems learn to performtasks by considering a lot of examples.

The neural network of the ADM 121 is configured to learn to reconstructeach data signal of the data 122 from an equipment of the system 110,based on all other data signals of the data 122.

Based on the difference between the reconstruction of a data signal andthe actual signal, abnormal sensor behavior can be detected, asdiscussed in the description.

Uniqueness of the embodiments are based on the fact that, in general,the approach to anomaly detection is different. Essential feature is touse a neural network within the ADM 121 to reconstruct (historical) dataof the data 122 and compare it to actual data of the data 122 with thegoal of detecting deviations (abnormal, anomalous behavior) in themarine vessel 105. This will be discussed more in relation to followingfigures and associated description.

In an embodiment, for the marine vessel 105 related anomaly detection,top priority for detection may be defined to be safety, and second andthird priority can be set by the ship operator (fuel consumption,speed/time, etc.), for example. The ADM 121 operates as a virtual expertfor providing insight on anomalities.

The fuel system 160 may be configured to select from at least one of thefollowing energy sources: diesel, liquified natural gas (LNG), liquifiedpetroleum gas (LPG), methanol, low sulphur heavy fuel oil (HFO), marinegas oil (MGO), and hydrogen, for example.

Propulsion system 150 may utilize power source to be selected from atleast one of the following: combustion-engine based power source; hybridpower source; and full electric power source. The propulsion system 150may comprise at least one thruster unit. Sensor data may be receivedfrom a plurality of sensors arranged to different parts of thepropulsion system 150. A sensor may be operationally connected to ashaft of the propulsion system 150, and another sensor may beoperationally connected to a blade of thruster unit, for example. Thustime-domain vibration data may be received from different parts of thepropulsion system 150.

The ADM 121 solution will allow different levels of automation withinvessels. In first operation mode, ADM 121 may be configured to providean anomaly detection plan, which the engineers can use for schedulingtheir activities. In second operation mode, ADM 121 may be configured toprovide an embedded solution, wherein the sub-systems can notify theoperator based on the anomaly detection plan, when to perform certaintasks or be switched on or set to standby. This notification may berepeated on the main display in the engine control room orremote-control station. In third operation mode, ADM 121 may beconfigured to provide a solution to be fully automated and automaticallyexecuting the anomaly detection plan of the ADM 121 with merelynotification provided to the operator or remote-control station whenperforming different automated tasks.

Furthermore, characteristic information representing at least oneoperating characteristic of the marine vessel 105 may be received asdata 122. The dynamic ADM 121 may use any available internal or externaldata such as weather forecasts and route plan information, and thevessel related data including the characteristic information.

In an embodiment, the anomality detection model 121 may be configured toperform a computer implemented method in a marine vessel data system,the method comprising: receiving time-domain data from a sensorconfigured to measure vibration of a system comprising a plurality ofoperationally connected parts, and operationally arranged to the marinevessel; generating a Fast Fourier Transform (FFT) on the time-domaindata to provide a plurality of spectra files in frequency-domain,wherein each spectra file comprises spectra data defined by amplitudeinformation and frequency information, and each spectra file isassociated with condition information determined based on collection ofthe time-domain data; normalizing each spectra file by converting thefrequency information to order information using the conditioninformation to provide normalized spectra files; generating actualsensor data based on the normalized spectra files; generating predicteddata for the at least one sensor as output by a convolutionalautoencoder type of neural network, wherein the convolutionalautoencoder type of neural network is configured to be trained usingnormalized reference spectra files, wherein each normalized referencespectra file comprises spectra data defined by amplitude information andorder information, and each spectra file is associated with timestampinformation determined based on collection of the reference sensor data;combining the predicted data with the actual sensor data for the sensorto provide error data; and generating 3D anomaly heatmap information,wherein a first dimension is defined by the timestamp information, asecond dimension is defined by the order information and a thirddimension is defined by anomaly information determined based on theerror data.

In an embodiment, the at least one sensor is configured to measurevibration of a component installed to the marine vessel. The at leastone sensor may comprise an accelerometer.

In an embodiment, the at least one sensor is configured to measurevibration of a propulsion system comprising a plurality of operationallyconnected parts.

In an embodiment, the condition information may comprise rotations perminute (RPM) information.

In an embodiment, the spectra data comprises data points configured todetermine the amplitude information as a function of the frequencyinformation or the order information. Each data point may be associatedwith the condition information according to the condition informationassociated to the spectra file.

FIG. 2 presents an example block diagram of a control apparatus 120within a marine vessel, for example, in which various embodiments of theinvention may be applied.

The general structure of the apparatus 120 comprises also devices suchas GPS 270 and at least one sensor 260.

The general structure of the control apparatus 120 may comprise a userinterface 240, a communication interface 250, a satellite positioningdevice (GPS) 270, capturing/sensor device(s) 260 for capturing vibrationdata, current activity data and/or current environmental data relatingto the vessel, a processor 210, and a memory 220 coupled to theprocessor 210. The control apparatus 120 further comprises software 230stored in the memory 220 and operable to be loaded into and executed inthe processor 210. The software 230 may comprise one or more softwaremodules and can be in the form of a computer program product. Thecontrol apparatus 120 may further comprise a user interface controller280.

The processor 210 may be, e.g., a central processing unit (CPU), amicroprocessor, a digital signal processor (DSP), a graphics processingunit, or the like. FIG. 2 shows one processor 210, but the apparatus 120may comprise a plurality of processors.

The memory 220 may be for example a non-volatile or a volatile memory,such as a read-only memory (ROM), a programmable read-only memory(PROM), erasable programmable read-only memory (EPROM), a random-accessmemory (RAM), a flash memory, a data disk, an optical storage, amagnetic storage, a smart card, or the like. The apparatus 120 maycomprise a plurality of memories. The memory 220 may be constructed as apart of the apparatus 120 or it may be inserted into a slot, port, orthe like of the apparatus 120 by a user. The memory 220 may serve thesole purpose of storing data, or it may be constructed as a part of anapparatus serving other purposes, such as processing data. A proprietaryanomaly detection application (client application) 231 comprising theanomaly detection model ADM 121 (or ADM data collection point in casethe ADM is arranged remotely at server apparatus) is stored at thememory 220. Propulsion system data, engine data, vibration data, sensordata and environmental data may also be stored to the memory 220. Theprogram code 230 may comprise the dynamic anomaly detection model (ADM)121 and the proprietary application 231 may comprise a clientapplication for the ADM, for example.

The user interface controller 280 may comprise circuitry for receivinginput from a user of the apparatus 120, e.g., via a keyboard, graphicaluser interface shown on the display of the user interfaces 240 of thecontrol apparatus 120, speech recognition circuitry, or an accessorydevice, such as a headset, and for providing output to the user via,e.g., a graphical user interface or a loudspeaker.

The satellite positioning device 270 is configured to provide locationinformation or time information, for example. Such information maycomprise, for example, position coordinates, speed, direction ofmovement, GPS time, and altitude information.

The communication interface module 250 implements at least part of datatransmission. The communication interface module 250 may comprise, e.g.,a wireless or a wired interface module. The wireless interface maycomprise such as a WLAN, Bluetooth, infrared (IR), radio frequencyidentification (RF ID), GSM/GPRS, CDMA, WCDMA, LTE (Long TermEvolution), or 5G radio module. The wired interface may comprise such asuniversal serial bus (USB) or National Marine Electronics Association(NMEA) 0183/2000 standard for example. The communication interfacemodule 250 may be integrated into the apparatus 120, or into an adapter,card or the like that may be inserted into a suitable slot or port ofthe apparatus 120. The communication interface module 250 may supportone radio interface technology or a plurality of technologies. Theapparatus 120 may comprise a plurality of communication interfacemodules 250.

A skilled person appreciates that in addition to the elements shown inFIG. 2, the apparatus 120 may comprise other elements, such asmicrophones, extra displays, as well as additional circuitry such asinput/output (I/O) circuitry, memory chips, application-specificintegrated circuits (ASIC), processing circuitry for specific purposessuch as source coding/decoding circuitry, channel coding/decodingcircuitry, ciphering/deciphering circuitry, and the like. Additionally,the apparatus 120 may comprise a disposable or rechargeable battery (notshown) for powering when external power if external power supply is notavailable.

In an embodiment, the apparatus 120 comprises speech recognition means.Using these means, a pre-defined phrase may be recognized from thespeech and translated into control information for the apparatus 120,for example.

The satellite positioning device 270 and the sensor device(s) 260 may beconfigured to be comprised by the apparatus 120 or connected as separatedevices to the apparatus 120. In case the satellite positioning device270 and the capturing device 260 are comprised in the apparatus 120 theymay be connected to the apparatus 120 using an internal bus of theapparatus 120. In case the satellite positioning device 270 and thesensor device 260 are external devices connected to the apparatus 120they may be connected to the apparatus 120 using communication interface250 of the apparatus 120 or using the internal bus.

The sensor device(s) 260 may be configured to, for example, measurevibration data, propulsion performance or operational data.

In an embodiment, at least one sensor device(s) 290 may be configurednot to be integrated to the marine vessel's information system 200 butconnected to the propulsion system 150 (see FIG. 1) only.

In an embodiment, sensor device(s) 290 are configured to be integratedto the propulsion system 150. No matter a single sensor 290 is shown,the sensor 290 may comprise a plurality of sensors 290. The sensordevices 290 may be configured to, for example, measure vibration data,propulsion performance or operational data.

In an embodiment, a communication interface (see e.g., FIG. 3) of thesensor device 290 comprises an automatic identification system (AIS)receiver for receiving a wireless transmission comprising automaticidentification system data, e.g. from the marine vessel. The AISreceiver may include an antenna configured to receive the automaticidentification system data or the sensor device 290 may include anantenna configured to receive the automatic identification system data.In another example, AIS receiver is configured to receive the automaticidentification system data from an antenna external to the sensor device290.

In an embodiment, the propulsion system 150 and at least one sensordevice 290 is configured to generate sensor data items based on thereceived automatic identification system data and sensor data. Thesensor data item may thus comprise sensor data generated by the sensordevice 290 and an identifier information. The identifier information maycomprise at least one of the following: sensor-ID (S-ID); propulsion-ID(P-ID), and vessel-ID (V-ID) that comprises at least part of thereceived automatic identification system data.

The sensor data item may also comprise information identifying themarine vessel (for example, International Maritime Organization (IMO)ship identification number or Maritime Mobile Service Identity (MMSI)).This identifying information may be taken from the automaticidentification system (AIS) signal or it may be stored within theapparatus 120 when installed.

The sensor data relating to the propulsion system and vibration measuredby the at least one sensor device 260, 290 may comprise measured datavalues as they were measured and/or data after processing at least someof the measured data values first.

In an embodiment, universal clock information of the control unit 200 isdetermined based on a vessel receiver device, comprising at least one ofthe Global Positioning System (GPS) receiver 270 and a communicationinterface 250 of the marine vessel. The universal clock information maycomprise at least one of the following: a Global Positioning System(GPS) time and a Coordinated Universal Time (UTC).

In an embodiment, a proprietary anomaly detection application (clientapplication) 231 is maintained at the apparatus 120. The application 231may comprise the ADM model 231 or the ADM model related data, such asanomaly detection data that is received from the server apparatus wherethe ADM model is run, for example. Anomalities are then determined usingthe ADM model 230, 231 (item 121 in FIG. 1) at control apparatus or atremote server apparatus if the ADM model is located there using theinput data.

No matter a plurality of elements is shown, all elements are notessential for all embodiments. Some elements are optional, such as GPS270, sensor device 290, user interface 240 and user interface controller280.

FIG. 3 presents an example block diagram of a sensor device 260, 290 inwhich various embodiments of the invention may be applied. The sensordevice 260, 290 may comprise various means for vibration data detection,activity data detection, operational data detection and environmentaldata detection, for example. The sensor device 260, 290 may be usedpropulsion system related data capturing or environmental datacapturing.

In an embodiment, the sensor device 260, 290 and the processing of thesensor data may provide a plurality of parameters relating to apropulsion system, for example one or more of the following: magnitudeand direction of proper acceleration, orientation, coordinateacceleration, shock, time, position, vibrations in three dimensions, andpropeller/engine RPM, for example. The operation conditions of thepropulsion system may comprise at least one of the following: number ofstarts; operating hours since last service; load cycles; miles traveledwith the propulsion system; amount of fuel used; and integral sensordata of the engine to provide engine sensor data, such as temperatureinformation, pressure information and mass flow information, forexample.

The sensor device 260, 290 may also comprise several capturing devices,combinations of any above-mentioned devices, and the like. Theenvironmental temperature may comprise air temperature, watertemperature or ground surface temperature, for example.

The sensor device 260, 290 may comprise also a communication interfacecapable of connecting with the communication interface 250 of FIG. 2.The generated sensor data may be transmitted to the communicationinterface of FIG. 2. The sensor device 260, 290 may also transmit itssensor data via its internal communication interface to thecommunication interface 250.

In an embodiment, the communication interface within the sensor device260, 290 is, for example, a wireless transmitter or a wirelesstransceiver (for example, Wireless Local Area Network (WLAN) transceiveror any mobile or cellular communication network transceiver (forexample, Wideband Code Division Multiple Access (WCDMA), Long TermEvolution (LTE) etc.), 5G or a local data port (e.g. Ethernet, UniversalSerial Bus (USB) etc.).

In an embodiment, the sensor device 260, 290 may also store informationidentifying the propulsion system or the marine vessel. This may havebeen preconfigured to the sensor device 260, 290. Since the automaticidentification system data identifies e.g. the marine vessel to whichthe received data relates to, the sensor device 260, 290 is thus able tomake sure that the received automatic identification system data relatesto the marine vessel to which the sensor device 260, 290 is affixed. Onepossibility for identifying the correct marine vessel is to use, forexample, wireless signal strength of the AIS signal. The strongest AISsignal relates to the marine vessel to which the sensor device 260, 290is attached. Yet another possibility is to compare the accelerationsignal from the acceleration sensor to the data indicating vesselmovements in the AIS signals and to determine the correct AIS signalbased on the comparison.

In an embodiment, a sensor device 260, 290 is configured to measure themarine vessel performance related data when the sensor device 260, 290is affixed to the hull structure of the marine vessel 105 of FIG. 1. Forexample, bolting, gluing or any other way for affixing or attaching thesensor device 260, 290 to the hull structure, propulsion element orengine body may be used. In other words, since the sensor device 260,290 is firmly attached to the hull, propulsion or engine structure,there is no relative motion between the sensor device 260, 290 and thestructure, and thus the sensor or sensors 260, 290 sense the motions andvibrations.

In an embodiment, an AIS receiver (comprised by the communicationinterface within the sensor device 260, 290 of FIG. 3) may receive awireless transmission comprising an AIS signal from the same marinevessel to which the sensor device 260, 290 is affixed. The sensor device260, 290 may beforehand store information identifying the marine vessel(for example, International Maritime Organization (IMO) shipidentification number or Maritime Mobile Service Identity (MMSI)) sothat it is able to determine that the AIS signal relates to the marinevessel 105 to which it is affixed. The AIS signal includes severalpieces of information relating to the marine vessel, for example, thevessel's identity, engine(s) identifier(s) and type, position, course,speed, navigational status and other related information. The sensordevice 260, 290 may utilize the AIS signal as it was received (in otherwords, every piece of information contained in the AIS signal). Inanother example, the sensor device 260, 290 may select a subset ofinformation included in the AIS signal to be included in the sensordata. In one example, the subset includes at least position and/or timeinformation of the marine vessel.

Normally the AIS signal is intended to assist a vessel's watch standingofficers to track and monitor movements of other vessels and allowmaritime authorities to track and monitor movements of vessels. It alsoidentifies and locates vessels by electronically exchanging data withother nearby ships.

In an embodiment, the AIS signal is received by a sensor device 260, 290installed in a vessel that is sending the AIS signal. This makes itpossible for the sensor device to link the AIS signal with sensor datameasured by the sensor or sensors 260, 290. Since the sensor device 260,290 has the information included in the AIS signal and measurements fromone or more sensors 260, 290, there is no need to make the traditionalintegration tasks to the marine vessel's information systems. The AISsignal sent by the marine vessel to the sensor device 260, 290 is astrong signal. Therefore, it may not be necessary to install a separateantenna in order to be able to receive the AIS signal. This makes theinstallation of the sensor device 260, 290 simpler and quicker. Forexample, it is possible to install the sensor device 260, 290 includingonly an internal antenna inside a marine vessel because the AIS signalleaks to the interior of the marine vessel via various existing cables,for example.

In an embodiment, a camera is configured to provide video signal asinput data for the dynamic anomaly detection model (ADM) 121 (FIG. 1),230 (FIG. 2). The camera may be regarded as sensor 260, 290 in relationto FIG. 2 or 3. Based on the video signal the apparatus may determine atleast part of the environmental or operational data. The determinationmay be done by video image processing, pattern recognition, filtering orother such means, for example. Alternatively, or additionally, directsensor data or online data (such as weather information) may be used.

The sensor device 260, 290 disclosed in FIGS. 2-3 may include at leastone accelerometer or three-dimensional accelerometer. Since the sensordevice 260, 290 may be affixed to the propulsion system, or the hull ofthe marine vessel or the engine body, the accelerometer is able to sensevibrations. From the vibrations sensed by the accelerometer, it ispossible to determine, for example, speed of rotation of a propeller ofthe marine vessel or of the main engine. In most vessels, the speed ofrotation of the propeller is identical with the speed of rotation of anengine of a marine vessel. Thus, it is possible to determine, based onan analysis of the measurements of the accelerometer, the speed ofrotation of a propeller and an engine of a marine vessel.

In an embodiment, in order to determine the speed of rotation of thepropeller, the sensor device 260, 290 or the associated computer devicemay analyze the signals measured by the accelerometer to identify thefundamental frequency in the signals. The fundamental frequency is theRPM (Revolutions Per Minute) of the engine or its multiple. One possiblemethod for pitch detection (i.e. find the fundamental frequency) is theHarmonic Product Spectrum (HPS) method. In the method, a spectrum iscompressed a number of times (down sampling), and it is compared withthe original spectrum. It can then be seen that the strongest harmonicpeaks line up. The first peak in the original spectrum coincides withthe second peak in the spectrum compressed by a factor of two, whichcoincides with the third peak in the spectrum compressed by a factor ofthree. Hence, when the various spectrums are multiplied together, theresult will form a clear peak at the fundamental frequency. It isobvious that the HPS is only one possible method for finding thefundamental frequency and other methods may be used. The speed ofrotation of the propeller may also be stored in the memory of the sensordevice 260, 290 to be transmitted to or accessed by an external entity.Related data may be used also as input for the dynamic anomaly detectionmodel (ADM) 121.

Furthermore, device or apparatus analyzing the sensor data, such as aserver apparatus, the sensor device itself, the control unit 200 or theremote apparatus may perform frequency analysis of the signals measuredby at least one acceleration sensor of the sensor device 260, 290. Incase the sensor device or some other device or apparatus performs thefrequency analysis, the amount of sensor data to be transmitted outsidethe sensor device/computer device is reduced. The frequency analysis maycomprise, for example, frequency-time analysis, such as Short-TimeFourier Transform (STFT) or Discrete Wavelet Transform (WFT). With thefrequency analysis, an understanding of frequency components over ashort time is received. The frequency analysis is performed, forexample, so that parameters of a marine vessel 105 (see FIG. 1) can beunderstood better and analyzed.

Further, the frequency analysis may comprise applying a dimensionalityreduction method, for example, Principal Component Analysis (PCA) inorder to identify the most significant components in the frequencydomain.

An accelerometer and an inclinometer can be used to measure the sameparameters since both measure acceleration. One of the main differencesis that the accelerometer provides acceleration components separately,but they are more inaccurate. However, acceleration components areusually provided within a larger dynamic range. The inclinometermeasures inclination more accurately but within a narrower range.Therefore, it is possible to perform RPM measurements also with theinclinometer if its bandwidth is high enough. Further, it may bepossible to perform a frequency analysis for the data provided by theinclinometer and get the same or almost the same results than based onaccelerometer data. One difference, however, is that the inclinometerdoes not measure vertical acceleration.

Based on the analysis of the sensor data, it may be possible todetermine the operation efficiency of the marine vessel (or the powerplant, for example) and its engine and to automatically trigger servicerequests such as oil change for the engine 291, for example.

Based on the information available and generated by the sensor device260, 290 it may be possible to optimize and analyze various factorsrelating to the marine vessel 105 (see FIG. 1), such as preventingunplanned downtime, extending maintenance interval, or improving energyefficiency of a marine vessel using the ADM model 230 (ADM 121 in FIG.1).

In an embodiment, data is received from a plurality of sensors 260, 290configured to measure vibration and operationally arranged to the marinevessel to provide time-domain reference sensor data, the time-domainreference sensor data is maintained within a data storage system, a FastFourier Transform (FFT) is generated on the time-domain reference sensordata to provide a plurality of reference spectra files infrequency-domain, wherein each reference spectra file comprises at leastcondition information and timestamp information associated to collectionof the time-domain reference sensor data, each reference spectra file isthen normalized by converting frequency to order information using thecondition information (RPM) to provide normalized reference spectrafiles, and a convolutional autoencoder type of neural network is thentrained using the normalized reference spectra files.

Based on input data and the trained ADM model (neural network based) itis possible to detect anomalities and further provide triggers for tasksor determine automated performance optimization, for example. Sensordetected vibration data may also be transmitted to the server apparatusand utilize the data and the model there for determining anomalities ortriggering customer related services, such as calculation of optimizedservice time, for example.

FIG. 4 shows a schematic picture of a computer implemented method andsystem for training a neural network based ADM model 400 according to anexample embodiment. The trained ADM model 400 may also be used forgenerating anomaly information, such as 3D anomaly heatmap information.The anomaly detection model (ADM) 400 may be configured to be maintainedat a remote server apparatus, at a remote control apparatus, at a marinevessel control apparatus or any combination of those. For example, datacollection 410 may be arranged from one apparatus (e.g. marine vesselcontrol apparatus), an insight input from a second apparatus (e.g.remote control apparatus) and the actual data processing and neuralnetwork involvement at a third apparatus (e.g. remote server apparatus),for example.

First, data is received from at least one sensor 410 or from a pluralityof sensors 410 configured to measure vibration and operationallyarranged to a marine vessel to provide time-domain reference sensordata. The sensor 410 may be configured to provide also frequency-domainreference sensor data.

The time-domain reference sensor data (and/or the frequency-domainreference sensor data, see above) is maintained within a data storagesystem 420. For example, at least one sensor may be operationallyarranged to a propulsion system of the marine vessel.

The data storage system 420 may be arranged to a remote serverapparatus, such as cloud server, but may also be at least temporarilystored at marine vessel data storage. The maintained sensor data of thedata storage system 420 is provided as source data for furtherprocessing.

Second, a Fast Fourier Transform (FFT) 425 is generated on thetime-domain reference sensor data to provide a plurality of referencespectra files 430 in frequency-domain, wherein each reference spectrafile 430 comprises spectra data defined by amplitude information andfrequency information, and each spectra file is associated withcondition information determined based on collection of the time-domainreference sensor data.

Third, each reference spectra file 430 may be normalized 435 byconverting frequency information to order information using thecondition information (RPM) to provide normalized reference spectrafiles 430 n.

In an embodiment, normalization step 435 may comprise several sub-steps.For example, a first sub-step may comprise that instead of indicatingthe specific frequency it is more advantageous for vibration analysis toknow the frequency relative to the input speed. To achieve this,frequencies are converted into orders. A second sub-step may comprise tolimit on specific orders which correspond to parts of the thruster. Forexample, data of approximately the first 40-50 orders could be selected.A third sub-step may comprise interpolation, wherein if we take intoconsideration that the availability of data in a certain period can below. To solve this complication data may be interpolated to a limitedrange of orders with an explicit number of new data points. The numberof resampled data points can be made with respect to the desiredprecision for the analysis of the data. As an example, three data pointsfor a range of 0.1 order could be used. These sub-steps are moredetailly discussed in connection with FIG. 10c , for example.

Fourth, a convolutional autoencoder type of neural network 440 istrained using the normalized reference spectra files 430 n.

Alternatively, other types of neural network 440 may be used, such asDenoising autoencoder, LSTM Autoencoder or Variational auto encoder, forexample.

FIG. 5 shows a schematic picture of a computer implemented method andsystem for generating anomaly information using a neural network basedADM model 400 according to an example embodiment.

The trained ADM model 400 may also be used for generating anomalyinformation, such as 3D anomaly heatmap information. The anomalydetection model (ADM) 400 may be configured to be maintained at a remoteserver apparatus, at a remote control apparatus, at a marine vesselcontrol apparatus or any combination of those. For example, datacollection 410 may be arranged from one apparatus (e.g. marine vesselcontrol apparatus and the actual data processing and neural networkinvolvement at another apparatus (e.g. remote server apparatus), forexample.

First, data is received data from a sensor 410 configured to measurevibration of a system comprising a plurality of operationally connectedparts, and operationally arranged to the marine vessel to provide actualsensor data 460.

The actual sensor data 460 may be provided by receiving time-domain datafrom a sensor 410 configured to measure vibration of a system comprisinga plurality of operationally connected parts, and operationally arrangedto the marine vessel, generating a Fast Fourier Transform (FFT) 425 onthe time-domain data to provide a plurality of spectra files 430 infrequency-domain, wherein each spectra file 430 comprises spectra datadefined by amplitude information and frequency information, and eachspectra file 430 is associated with condition information determinedbased on collection of the time-domain data, normalizing each spectrafile 430 by converting the frequency information to order informationusing the condition information to provide normalized spectra files 430n, and generating actual sensor data 460 based on the normalized spectrafiles 430 n.

The time-domain data may be received directly from the sensor 410without first maintaining at the data storage system 420 or firstmaintaining at storage 420. The data storage system 420 may be used tomaintain also reference sensor data used for training the model 400. Thereference sensor data may be data collected for actual sensor data 460received over time, and used as history data for training the model, forexample.

Second, predicted data 450 is generated for the at least one sensor 410as output by a convolutional autoencoder type of neural network 440,wherein the convolutional autoencoder type of neural network 440 isconfigured to be trained using normalized reference spectra files 430 n,wherein each normalized reference spectra file 430 n comprises spectradata defined by amplitude information and order information, and eachspectra file is associated with timestamp information determined basedon collection of the reference sensor data. Timestamp information maycorrelate to time information (year/month/date) when the sensor data wasreceived by the sensor 410, either for reference data use to train themodel or to be used as actual sensor data 460.

Third, the predicted data 450 is combined with the actual sensor data460 for the sensor to provide error data 470.

Fourth, 3D anomaly heatmap information 480 is generated, wherein a firstdimension is defined by the timestamp information, a second dimension isdefined by the order information and a third dimension is defined byanomaly information determined based on the error data 470.

In an embodiment, consecutive spectra files 430-430 n of the actualsensor data 460 may be detected in the first dimension with anomalyinformation in the third dimension, wherein the anomaly informationexceeds pre-defined anomaly threshold, at least one detected spectrafile may be determined to identify order information for the anomalyinformation, the identified order information may be compared with partfrequencies of the plurality of operationally connected parts (that thesensor 410 is operationally connected to), and faulty part may bedetermined based on the comparison.

In an embodiment, order information may be identified with at least oneharmonic for the anomaly information when determining the at least onedetected spectra file, the identified order information may be comparedwith the at least one harmonic and the part frequencies of the pluralityof operationally connected parts, and faulty part determined based onthe comparison.

In an embodiment, anomalies may be detected in a sub-range of the seconddimension based on the 3D anomaly heatmap information 480, and second 3Danomaly heatmap information 490 may be generated, wherein a firstdimension is defined by the timestamp information, a second dimension isdefined by the order information of the sub-range and a third dimensionis defined by anomaly information determined based on the error data470.

In an embodiment, false anomalies may be determined based on the second3D anomaly heatmap information 490 by identifying order pole passfrequencies.

In an embodiment, at least one spectra file 430-430 n of the actualsensor data 460 may be detected with anomaly information in the thirddimension, wherein the anomaly information exceeds pre-defined anomalythreshold, order information may be determined for the anomalyinformation in the sub-range, the determined order information comparedwith order pole pass frequencies and part frequencies of the pluralityof operationally connected parts, and faulty part determined based onthe comparison. Order information relating to pole pass frequencies maybe ignored when determining faulty part.

Error data 470 generation may be part of the ADM model 400 and 3Dheatmap information 480 may also be generated by the model 400 no matterit illustrated as being generated outside the model 400.

In an embodiment, the sensor 410 is configured to measure vibration of apropulsion system comprising at least a shaft, a bearing, a propellerbase and a propeller blade, and at least one of them having pre-definedpart frequency provided for determining faulty part.

The condition information may comprise rotations per minute (RPM)information. Vibration data may be thus received through a single sensormeasuring vibration relating to different parts of the propulsionsystem, such as a thruster.

In an embodiment, the convolutional autoencoder type of neural network440 is arranged at a remote server apparatus and the sensor 410 or theplurality of sensors 410 are operationally arranged to an engine systemor a propulsion system of the marine vessel. Generating the predicteddata 450 may comprise reconstructing data of at least one sensor 410, bythe neural network 440, based on the sensor data 420.

In an embodiment, for training the model, data is received from at leastone sensor 410 configured to measure vibration and operationallyarranged to a marine vessel to provide time-domain reference sensor dataor frequency-domain reference sensor data, depending on the arrangement.

The time-domain reference sensor data or the frequency-domain referencesensor data is maintained within a data storage system 420. For example,a sensor may be operationally arranged to a propulsion system of themarine vessel, wherein the propulsion system comprises a plurality ofoperationally interconnected parts, like shaft, bearing, base andpropeller blade, for example.

The data storage system 420 may be arranged to a remote serverapparatus, such as cloud server, but may also be at least temporarilystored at marine vessel data storage. The maintained sensor data of thedata storage system 420 is provided as source data for furtherprocessing.

A Fast Fourier Transform (FFT) 425 may be generated on the time-domainreference sensor data to provide a plurality of reference spectra files430 in frequency-domain, wherein each reference spectra file 430comprises spectra data defined by amplitude information and frequencyinformation, and each spectra file is associated with conditioninformation determined based on collection of the time-domain referencesensor data.

Each reference spectra file 430 may be normalized 435 by convertingfrequency information to order information using the conditioninformation (RPM) to provide normalized reference spectra files 430 n.

In an embodiment, normalization step 435 may comprise several sub-steps.For example, a first sub-step may comprise that instead of indicatingthe specific frequency it is more advantageous for vibration analysis toknow the frequency relative to the input speed. To achieve this,frequencies are converted into orders. A second sub-step may comprise tolimit on specific orders which correspond to parts of the thruster. Forexample, data of approximately the first 40-50 orders could be selected.A third sub-step may comprise interpolation, wherein if we take intoconsideration that the availability of data in a certain period can below. To solve this complication data may be interpolated to a limitedrange of orders with an explicit number of new data points. The numberof resampled data points can be made with respect to the desiredprecision for the analysis of the data. As an example, three data pointsfor a range of 0.1 order could be used. These sub-steps are moredetailly discussed in connection with FIG. 10c , for example.

A convolutional autoencoder type of neural network 440 may be trainedusing the normalized reference spectra files 430 n.

Alternatively, other types of neural network 440 may be used, such asDenoising autoencoder, LSTM Autoencoder or Variational auto encoder, forexample.

The predicted data 450 may be combined with actual sensor data 460 forthe sensor or the subset of the plurality of sensors to provide errordata 470, wherein the actual data 460 is processed based on sensor datareceived from a sensor 410 or is processed based on sensor datamaintained at the data storage system 420 based on the sensor ID of theplurality of sensors.

Actual sensor data 460 may be processed in corresponding way from thesensor 410 through steps 420-430 n all the way to be combined to provideerror data 470.

In an embodiment, generating the predicted data 450 comprisesdetermining correlation, by the neural network 440, between the datafrom the plurality of sensors 410. Correlation may be determined, by theneural network 440, for the sensor 410 or the subset of the plurality ofsensors 410.

In an embodiment, the neural network 440 is trained by means of signalsfrom the individual sensor 410 for determining internal neural networkparameters. The neural network 440 is used for determination of theanomaly 480, 490 based on the error data 470.

In an embodiment, the neural network 440 is configured to simulate thepredicted data 450, and the neural network 440 is adjusted to the sensordata 420 by means of a training function. The training function maycomprise, for example, at least one of the following: Gradient descentfunction; Newton's method function; Conjugate gradient function;Quasi-Newton method function; and Levenberg-Marquardt function.

In an embodiment, remote normalized reference spectra files 1730 noriginating from a remote apparatus 1730 may be received and theconvolutional autoencoder type of neural network 440 may be trainedusing the remote normalized reference spectra files 1730 n.

In an embodiment, feedback information 496 may be gathered by anevaluation application 495 continuously. If, for example, an anomalyexpert marks a period of data as irrelevant, it may be configured todisappear at the view of all other experts, so nobody need to check thatperiod of data anymore. Then again, if a stakeholder of a case closesthe cases and marks the case as highly severe, a separate model maylearn this so that future similar anomalies will be more prominentlyindicated heatmap information 480, 490. This will lead to the expertsvalidating the data earlier, thereby shortening the time between modeloutput and case creation. This would benefit the stakeholders as thisenables information to be available faster to them.

The insight input 495 may be received from a second apparatus (e.g.remote control apparatus) and the actual data processing and neuralnetwork involvement at a third apparatus (e.g. remote server apparatus),for example.

FIG. 6 shows a schematic picture of an autoencoder architecture 600 of aneural network configured to be trained using normalized referencespectra files according to an example embodiment.

Autoencoders are a class of symmetric neural networks used forunsupervised learning which learn to recreate a target. The differencewith autoencoders is that the output layer is of the same dimensionalitywith the input layer, there is not target value in this case since thegoal is to reconstruct the input without an explicit target value.Otherwise stated, the autoencoder attempts to learn the identityfunction by minimizing the reconstruction error. An autoencoder consistsof two parts, the encoder and the decoder. The encoder is a function fthat reads the input data Xi and compresses it to a latentrepresentation usually of lower dimensionality Z.

where f is the activation function of the encoder, Wf is the matrix orweights for the encoder, x is the input data Xi and bf is the biasvector for thez=f(x)=σ_(f)(W _(f) *x+b _(f))encoder. Next, the decoder will read the compressed representation Z andtry to recreate the input Xi with output Xo like illustrated in FIG. 6.

Accordingly, the decoder functions and output layer Xo are given by:{circumflex over (x)}=t(z)=σ_(t)(W _(t) *z+b _(t))

During the training phase, autoencoders attempt to find a set ofparameters theta=(W, bf, bt) that will minimize the loss function L.Once again, the loss is used as a quality metric for thereconstructions. Evidently the goal is, for output Xo to be as close aspossible to original input Xi.

The loss function will help the network to find the most efficientcompact representation of the relations in the training data, withminimum loss. As can be seen in FIG. 6, the number of neurons in thehidden layer Z are less than those of the input layer Xi. By compressingthe input, the neural network 440 (see FIGS. 4-5) will be forced todiscover the relations between the input features of the training datato be able to reproduce it.

Autoencoders can also be stacked by implementing layers that compresstheir input, to smaller and smaller representations. Afterwards, similarto encoding stacked layers are used for decoding. Deep autoencoders havegreater expressive power and the successive layers of representationscapture a hierarchical grouping of the input, similar to the convolutionand pooling operations in convolutional neural networks. Deeperautoencoders can learn new latent representations of the data, combiningthe ones from the previous hidden layers. Each hidden layer can be seenas a compressed hierarchical representation of the original data, andcan be used as valid featured describing the input. The encoder can beconsidered as a feature detector that will generate a compactsemantically rich representation of the input.

In its simplest form the autoencoder is a three-layer neural networklike in FIG. 6. There are numerous types of autoencoders though that canbe implemented depending on the problem at hand. This varies fromdenoising, convolutional, recurrent and most recently variationalautoencoders. Choosing which type of autoencoder to apply for eachproblem depends on the data that is being modeled.

In the traditional architecture of autoencoders, the fact that a signalcan be seen as a sum of other signals is not taken into account.Convolutional autoencoder (CAE), on the other hand, use the convolutionoperator to accommodate this observation. The convolution operatorallows filtering an input signal to extract some part of its content.CAE learns to encode the input in a set of simple signals and then tryto reconstruct the input from them. A CAE, like any autoencoder, isgenerally composed of two parts, corresponding to the encoder and thedecoder. By transforming the input into a lower dimensionalrepresentation, the model can learn the correlation between thedifferent data points, which in these embodiments means the spectradata. By training on normal data, without anomalies, the ADM model 400will learn the expected behavior and patterns expected from thevibrations.

By using the reconstruction error of the autoencoder, we will be able toobserve which frequencies differ significantly more than the expectedand determine any potential anomalies.

FIG. 7 shows a schematic picture of high level steps of the proposedapproach according to an example embodiment.

After data collection 710, prepossessing 720 will be applied to shape itinto an appropriate form for training. Finally, after using thepredictions of the autoencoder 730, the anomaly analysis 740 will takeplace.

Analysts often use vibration analysis to investigate a machine andmonitor its status for early warnings of fault conditions. For rotatingequipment this could be misaligned components, damaged bearings etc.

Vibration data may be derived by using the FFT on time domain signalscollected on board a vessel, for example.

All rotating machines such as fans, motors and turbines vibrate whenthey are operating. As each component rotates it emits a vibrationresponse at a certain frequency. As the speed of rotation changes, theresponse changes as well. All the different rotating forces within themachine cause vibration and can therefore be tracked. These forcesrelate to all rotating elements like the shaft, the ball within thebearing, the blades of the propeller etc.

To extract the vibration pattern from machinery, accelerometers may beused for monitoring the systems. Vibration is expressed in metric unitsm/s2, or in some instances, in units of gravitational constant g, whereg=9.8 m/s2. The vibration in this case is the mechanical oscillationabout an equilibrium position of a component. The accelerometer measuresthe dynamic acceleration as a voltage. For thrusters, which provide thedata set of this example, the accelerometers are typically directlymounted on high frequency emitting elements, like the bearings of theelectric motors. Rotations per minute (RPM) are used as the unit for theinput speed of the thruster.

FIG. 8 shows a schematic picture of using FFT for vibration analysisaccording to an example embodiment.

To illustrate how an FFT can be used for vibration analysis, an of acomponent may be analyzed, in this case a fan 810. The fan 810 consistsof two rotating components, a shaft and the blades, each with adifferent frequency and amplitude. Within one rotation of the shaft,there are seven repetitions for the blades. These two parts of the fanwill produce a composite waveform, also called overall vibration, thatlooks rather complex in the time domain. By converting the vibration tothe frequency domain using an FFT, the individual sine waves can be moreeasily identified, as they will show up as spikes at frequencies thatcorrespond to the rotating components.

Any composite waveform is the summation of multiple sinusoid signals ofdifferent frequencies, amplitudes, and phases. The FFT is used todeconstruct these composite complex waveforms into the individual sinewave components. The result is an amplitude function of the frequency,which allows an easier analysis in the spectrum (frequency) domain,compared to the more complicated signal of the time domain. This way itis possible to gain a deeper understanding of the vibration pattern andprofile.

Notice that in FIG. 8, the overall vibration signal of the fan is acombination of the vibration from the shaft and the blades. The fanrotates at a fixed RPM. The shaft rotates at the same rate as therotational speed of the fan, whereas the rotational speed of the bladesis higher than the one of the fan. The vibration signal of the shaft hasthe same frequency as the rotational speed of the fan, which correspondsto the first harmonic “shaft” of the right part of FIG. 8. The bladevibration signals have a higher frequency than the rotational speed ofthe fan, which corresponds to the vibration value.

The example of FIG. 8 is a simplistic case where the overall vibrationconsists of only two signals. In reality, the composite waveforms arecomposed of considerable more signals.

FIG. 9a shows a schematic picture of having three signals producing thetimeseries according to an example embodiment.

In FIG. 9a it can be seen how having three signals make the timeserieslook more complicated. This constructed waveform is composed of threefrequency components with values 22 Hz, 60 Hz and 100 Hz, with addedbroadband noise. This is closer to an example of a machine in real life,since we also notice noise in machinery equipment. This makes thesignals hard to distinguish and is not optimal for condition monitoringin thrusters. FIG. 9a kind of waveform is too complex for visualanalysis.

FIG. 9b shows a schematic picture of having three signals producing thefrequency series after applying FFT according to an example embodiment.

By using the FFT in in FIG. 9b , we can clearly distinguish the threefrequency components individually at their respective frequency value(22 Hz, 60 Hz, 100 Hz). Using an FFT it is possible to clearly identifythe major frequencies to determine the vibration signal. Note, that thistime it is also possible to detect the added noise in the rest of thespectra, which is indicated by low amplitude signals at otherfrequencies.

Complex machines like thrusters produce more complicated vibrationsignals, from various different sources, which result in a highlycomplex overall vibration. In practice, machines have several moresources of vibration. Since the goal of the analysis is to understandthe condition of the machine, it is desired to assess the vibrationrelated to the most common fault conditions of a component likemisalignment, unbalance or broken bearings.

The data that is used for different embodiments consists of numerousspectra files sampled for various dates. The availability of the spectrafiles, is inconsistent and filtering techniques, like low pass filtersare used to focus on useful information.

Each file contains, for example, 2622 different amplitude valuesoriginating from various vibrations sources, like the ones from FIG. 8.Every file available has various other information available in additionto the vibration amplitude values. This information may include, forexample:

Installation information: Each file has a tag name to identify for whichvessel this data has samples from.

StartTime/EndTime information (timestamp): Values corresponding to thetime of the sampling process. The format may be year/month/day.

Condition: This is the input speed value in RPM, of the thruster duringsampling.

Vibration amplitude: The main data used. This may of 2622 values foreach file of the data set.

FIG. 10a shows a schematic picture of one of the spectra files plottedwith respect to the corresponding frequency according to an exampleembodiment.

In the x-axis the corresponding frequency (0-1000 Hz) is visualized withy-axis showing the values for the corresponding amplitude (0.000-0.035m/s2).

There are a plurality of amplitude values (2622) for the total frequencyrange [0-1000] Hz. This is because during the data sampling, a low-passfrequency filter is used.

On the x-axis of the file in FIG. 10a the corresponding frequency ofeach vibration detected by the sensors is shown. When referring to theoccurrence of a repeating event it is convenient to do it in terms ofmultiples of running speed rather than absolute Hz. This is because ofvarying RPM values, which make the scale in frequency inconsistent fordifferent files. So instead of indicating the specific frequency it ismore advantageous for vibration analysis to know the frequency relativeto the input speed. To achieve this, frequencies are converted intoorders. It is advantageous to use orders for analysis of vibrationsignals, as they help with ignoring the noise of irrelevant rotatingcomponents.

If a vibration signal is equal to twice the input speed RPM from athruster, for example, then the order is two, with the first order beingthe input speed rotation value. By utilizing the orders it is possibleto track each individual component with more ease.

To obtain the orders we may use following equation:Order=Frequency(Hz)*60/Input speed(RPM)

It is worth pointing out that frequency is the number of events per unitof time, so by multiplying with 60 seconds it is possible to discoverthe number of events per minute.

Accordingly for each individual frequency f_i for [f_1, f_2622], whichare all the data points used for the RPM measurement for each file andcalculated the new axis in orders. For example to acquire the ordercorresponding for an amplitude value with fi and input speed s we haveequation:Order_i=f_i*60/s

where s is the input speed, for the respective file of 2622 data points,and is measure in RPM.

FIG. 10a can be used to calculate the orders for all 2662 data points.The converted x-axis, in orders, is visible in FIG. 10 b.

FIG. 10b shows a schematic picture of one of a spectra files after ordernormalization has been applied according to an example embodiment.

A spectra file after order normalization has been applied. Afterobtaining the orders it is easier to detect peaks corresponding to thecomponents of interest.

The input speed for this file was 600 RPM, thus using equation forOrder_i it is possible to get a maximum order value of 100. Note thatthe amplitudes are still the same, essentially only the x-axis labelsare different.

FIG. 10b may still seem quite complicated. Focus may be given onspecific orders which correspond to parts of the thruster, for example.Usually this would amount to approximately the first 40-50 orders, forexample.

To clarify the analysis of the data, the first part of the file may beanalyzed like the analysts and identify what the peaks correspond to.

FIG. 10c shows a schematic picture of one of a spectra files for thefirst 30 orders according to an example embodiment.

As shown in FIG. 10c the same file of FIG. 10b may be examined but thistime focus is given on the first 30 orders. Specifically it can be noteda number of interesting peaks 1010, 1020 at first and second order.Since the orders are now used, at the first order we always have thepeak at the running speed of the thruster, for example.

Using units of orders, it is therefore possible to find the source ofeach vibration in a more straightforward manner. At the 8th order we cansee the pole pass peak 1010 rising higher than the noise floor createdby the rest of the peaks. The noise floor can be thought as thehorizontal line set by the majority of the peaks, in this case about0.015 m/s2. The orders that correspond to different mechanical parts areknown and they are referred to as forcing frequencies.

As can be seen in FIG. 10c , at the 16th order, the first harmonic 1020of the pole pass frequency is visible. Harmonics are a series of evenlyspaced peaks that are multiples of any forced frequency, and are commonin periodic signals in vibration analysis. Locating all sets ofharmonics is of great importance during the analysis process becausethey verify that an anomaly is present, if both the fault frequency andthe first harmonic amplitudes are significantly high.

These peaks 1010, 1020 correspond to the input speed of the thruster andits first harmonic. The peaks visible at 8^(th) order 1010 and 16thorder 1020 correspond to the pole pass frequency. This is considered acommon pattern for the spectra files.

As explained, the input speed of each file, measured in RPM, is utilizedand thus the values for the orders are calculated. However, the thrusteris operating at different speed values, when the sampling process takesplace. The RPM values varies from file to file and this causes aninconsistency in the analysis of the files.

For example, an input speed of 600 RPM for the thruster will lead tomaximum order of 100, while a file with input speed of 180 RPM leads toa maximum order of approximately 330. This inconsistency is causedbecause the file of 180 RPM is spread across a broader scale of orderswhile the file of 600 RPM is more compressed due to the higher thrusterspeed during sampling.

More clearly the effect of the fluctuating RPM on the order scale ifdrawing to 3D graph where on the x-axis has the orders, the y-axis hasthe amplitudes and z-axis showcases the RPM for the different files.Naturally, files with higher RPM values would appear more condensed thanthose with lower RPM. Again, it should be mentioned that the data pointsare still the same size for all files. Now, each measurement of theavailable 2622, corresponds to a different order.

This irregularity for the data is troublesome. Since the current anomalydetection is based on comparing the latest data with the old, files thatare approximately of the same thruster speed are typically used. This iseven more complicated if we take into consideration that theavailability of data in a certain period can be low.

To solve this complication the data can be interpolated to a limitedrange of orders with an explicit number of new data points x_i with ibelongs to {1, . . . 1600}. The number of resampled data points can bemade with respect to the desired precision, necessary for the analysisof the data. After further study on data and the current analysistechnique, it was found that a precision of three data points for arange of 0.1 order is appropriate.

Therefore all files are cut off at the maximum of 50 orders and theninterpolated with 1600 points. xk=x1, x2, . . . , x2622→x_i=x_1, x_2, .. . , x_1600.

The new frequency orders f_i are of a fixed range with a discrete set ofvalues needed for analysis.

By implementing a limit of 50 orders and i belonging to {1, . . . ,1600} this leads to a step of approximately 0.03 order for each datapoint x_i.

This is important as it solves the challenge of inconsistent data pointfrequency. Since we want to use an autoencoder to learn the relationbetween the data points, we need a fixed input space for training.However, the input speed for each file differs.

This causes the amplitude values for the first 50 orders, which arerequired for analysis, to be of different amounts. This effect provesproblematic for the analysts, because in most instances the analysis hasto be for files with almost the same input speed. This limitation issolved by using a neural network 440 (see e.g. FIG. 4 or 5).

Before training of the neural network 440, it is possible to use, forexample, two different scaling methods to scale the data. This isadvantageous when dealing with datasets containing varying values andranges, like the one disclosed as an example. In cases of data featuresexpressed in great magnitude, in deep learning (DL) it is customary toscale the data before used for the models.

Minimum-Maximum Scaler

The Min-Max scaling method is considered a simpler scaler. This methodrescales the data in such a way that all values are in the range [0, 1].Following function provides the rescaled data points:x′=(xi−min(x))/(max(x)−min(x))

where x′ is the normalized values and x is the original value.

Robust Scaler

Robust scaler uses a similar approach to the Min-Max scaler. Instead ofusing the minimum and maximum vales of the data set, the robust scaleruses the interquartile ranges of the data, as noted by followingfunction:x′=(xi−Q1(x))/((Q3(x)−Q1(x))

where x′ is the normalized values, x is the original value and Q1, Q3are the first and third quartiles respectively. After experimenting withboth scalers, robust scaler was found to provide slightly betterresults.

Because the robust scaler is based on quartiles it is not stronglyinfluenced large outliers.

In an embodiment, after the data preprocessing step 720 (see e.g.prepossessing 720 in FIG. 7) is completed, the next step is to determinethe neural network 440 and train on the predefined training data set.The representation learning model is a convolutional autoencoder 600(see also e.g. element 730 in FIG. 7).

Input Layer Xi (see e.g. FIG. 6). The determined neural network 440.600, 730 receives as an input the vibration signal files. Each file istreated as a different training example for the model. The amplitudesare processed, before used as input, with the techniques discussedearlier, namely data interpolation and scaling. The input sequence isx=(x1, x2, . . . , xn) with n=[1, 2, . . . , 1600] interpolated datapoints, for example. This layer serves only to define the inputdimensionality of the data. In the example case of this solution it is[1, 1600] (one dimension, 1600 data points).

Convolution Layer: The convolution layer is the core of the CAE model.This is where the actual model learns the representation of the data.Input parameters include filters and kernel size. Integer values areused for the parameters which define the number of kernel windows andthe size of kernel for training, respectively.

Pooling Layer: This layer reduces the input dimension size of theoriginal data. There are different choices for pooling, with the mostcommonly used being Max pooling. Max pooling extracts the maximum valueof a window of the feature map, similar to the convolution method, asseen in FIG. 11.

FIG. 11 shows a schematic picture of max pooling according to an exampleembodiment.

FIG. 11 illustrates an example of 2 dimensional max pooling 1100. As canbe seen, for each different window 1110-1140 the maximum value isextracted.

Usually the window size, or pooling grid, used for pooling though issmaller with a typical size being 2. The neuron with the maximumactivation value in the window 1110-1140 is then extracted to resultingwindow 1150 and the rest are discarded. Another choice for pooling is bythe average method. Average pooling simply aggregates a region into theaverage values of the activation observed in that region.

Upsample Layer: The objective of the upsampling layer is to transformthe data to the original input dimensionality, by repeating values alonga certain axis.

Dropout Layer: Another important layer used is the Dropout. Thistechnique takes as input parameter a probability p, and discards, or asthe name suggests drops out, each neuron with a probability pstochastically. So then during the training of the neural network 440these neurons will be essentially ignored from the model along with allinput and output connections to other neurons. This helps preventoverfitting of the network on the training data.

In an embodiment, hyperparameter optimization may be needed to implementthe neural network 440 when there is a set of parameters that we need todefine. To determine the best performing set of parameters for themodel, different combinations can be tested and evaluated according tothe loss. The loss in this case is the main evaluation metric for theperformance of the neural network, as it essentially represents thequality of the reconstructions.

It is advantageous then to aim for the lowest loss it is possible toachieve, along with respect to the quality of the reconstructions.

In an embodiment, the choice of a loss function should match the framingof the specific prediction problem. In view of the embodiments, usingloss functions appropriate for regression problems should be preferred.As explained before, loss is the quantity that is minimized during thetraining of the neural network 440. For regression problems, like theproblem of the embodiments disclosed within vibration data anomalydetection, there are a number of appropriate loss functions that can beapplied.

In an embodiment, Mean Squared Error (MSE) is the default loss functionfor regression problems, dealing with real values. It measures theaverage squared difference between the predicted and target values asshown below:

${MSE} = {\frac{1}{n}{\overset{n}{\sum\limits_{i - 1}}( {y_{i} - {\hat{y}}_{i}} )^{2}}}$

Similar to MSE, Mean Absolute Error (MAE) is another loss for regressionmodels which measures the sum of absolute differences, the function isgiven by equation as shown below:

${MAE} = {\frac{1}{n}{\overset{n}{\sum\limits_{i - 1}}( {{y_{i} - {\hat{y}}_{i}}} )^{2}}}$

Finally, the logarithm of the hyperbolic cosine (Log cosh) loss functionmy be used. Log cosh works like MSE for small differences, while forhigher values it is similar to MAE. Hence, it is not strongly affectedby occasional incorrect predictions as much. The function is given byequation as below:

${{Log}\;\cosh} = {\frac{1}{n}{\underset{i = 1}{\sum\limits^{n}}{\log( {\cosh( {y_{i} - {\hat{y}}_{i}} )} )}}}$

After evaluating the loss functions mentioned, it was found that MSEprovides the best reconstruction results. The loss also convergences, asdesired for neural network 440. Both training and test loss converge,after the corresponding data sets have been through the model asufficient number of epochs. Increasing the number of epochs more than100 did not show any significant decrease in loss, so to avoidoverfitting a maximum number of 100 epochs is used for experiments.Having a higher, than necessary, number of epochs can cause the model tooverfit the training data.

In an embodiment, different activation functions can be used. Theactivation functions used for training comprise relu, tan h and sigmoid,for example.

The relu function is the most commonly used and recommended for feedforward neural networks, with a good performance. Relu is essentially apiece wise function of two linear pieces, and is mostly good forconvolutional neural networks where the data has a topologicalstructure.

In an embodiment, optimizer algorithms can be used to minimize the lossof the network which is dependent on the parameters that the modellearns, such as the weights. Optimizers serve as the mechanism throughwhich the network will update the weights, based on the data used asinput.

For example, three optimizers could be considered. First, AdaptiveMoment Estimation (Adam) is a method that computes adaptive learningrates for the model's parameters. Adam is computationally efficient andhas low memory requirements. It is optimization algorithm and comparesfavorably to other optimizers. Second, Root Mean Square Prop (RMSProp)and third, Stochastic Gradient Descent (SGD) could also be used.

Other parameters include the number of hidden layers Z (see FIG. 6) andnumber of kernel filters. The number of the hidden layers Z correspondsto the depth of the model 600.

There is no general rule of thumb for the correct number of the hiddenlayers Z, it is dependent on the complexity of the each problem. It hasbeen shown in studies that having unnecessarily many stacked hiddenlayers will cause a lower prediction accuracy. This is due to the factthat the model, having more than the sufficient number of layers, canoverfit to the data set used for training, thus failing to generalize tothe new unseen validation data set.

After obtaining the lowest losses, the lowest curves were selected andplotted to observe if the loss curves have converged. Afterwards, thereconstructions were observed to validate the performance of the model.It was found that actually for combinations C(3), C(4) and C(5) themodel would reconstruct all data points used for test, along withsimulated anomalies.

Table below shows validation loss values for different sets ofhyperparameters. After obtaining the parameters with the lowest loss,the predictions are also studied to decide on the optimal combination.

Combinations Layers Filters Kernel size Validation loss C(l) 1 32 80.0074 C(2) 1 8 8 0.0057 C(3) 1 8 32 0.0012 C(4) 1 32 32 0.0018 C(5) 2 832 0.0091

Using a high kernel size even though providing the lowest loss, wasdeemed not appropriate for anomaly detection due to the fact of the“mirroring” effect. This happens in cases where the autoencoder does notactually learn the data but memorizes all data points. Finally, betweenusing 8 and 32 filters, 8 seemed to deliver better results with respectto reconstructions.

In an embodiment, it is determined parameters of the optimal model. Thiscombination of parameters was found to be optimal combining a lowconverged validation loss, and valid predictions:

Loss Function MSE Number of Layers 1 Optimizer Adam Kernel Size 8 Numberof Filters 8 Activation Function Relu

The optimal set of parameters used to obtain the results is summarizedin table above to provide the most efficient results with a low loss.

In an embodiment, after the parameters of the model are finalized thefinal step is to analyze the quality of the reconstructions.

FIG. 12 shows a schematic picture of an example of a spectra filewithout any anomalies detected according to an example embodiment.

In FIG. 12 it is shown a spectra file with predictions, for the first 30orders. High amplitudes in order 8 illustrated with a peak 1210, inorder 16 illustrated with a peak 1220, and in order 24 illustrated witha peak 1230 are clearly visible and reconstructed. This is the pole passfrequency at 8th order along with the 1st and 2nd harmonic. Moreover, avibration in 12^(th) order illustrated with a peak 1240 is visible, thisis called a ghost frequency, and does not correspond to a specificcomponent but is not considered an anomaly either. The reconstructionsare the same with the original data. This file is thus considered cleanand of high quality.

Some file patterns do not imply that a thruster component is damaged,however it is data that the human analysts usually ignore and not takeinto account since the data points are irrelevant and do not offer validinformation.

Such sampling anomalies occur in cases when the positioning of thethruster is unstable or due to overheating of the sensors.

The model 400, 600 is able to understand that these files are abnormalsince the reconstructions are significantly different throughout thevibrations for all orders.

FIG. 13 shows a schematic picture of 3D heatmap information 1300 for anexample of a spectra file with anomalies detected according to anexample embodiment.

To evaluate the performance of the solution, anomalies were simulatedinto the available test dataset, to observe if the model would detectthem. The anomalies were created with respect to real problems that theanalysts face. Specifically high vibration amplitudes were introduced at8.6 order which represents the Ball Pass Outer Race (BPO) anomaly, and10.4 which represents the Ball Pass Inner Race (BPI). The peaks werecreated for five consecutive files, in a random point of the dataset toavoid a biased evaluation.

High peaks can be observed in these orders after they were introduced inthe original files used for testing, visualized in the figures tofollow. The amplitude values for the anomalies were calculated accordingto the maximum values for each file accordingly. The anomalies were alsoconfirmed to represent real life examples by the analysts. The orderscorresponding to the anomalies are called fault frequencies. In mostinstances, the anomalies are confirmed where an abnormally high peak isalso visible at the first harmonic of the fault frequency, hence in thecase of the BPO that is order 17.2. If the analysts observe that thesepeaks are higher than the baseline, they deduce that an anomaly ispresent in the data. Anomalies for the first harmonic of the BPO werethus also simulated. For the analysis of the proposed solution, in thecase of the anomalies, the goal for the model is to not reconstructthese peaks, since they are abnormal. The difference between theoriginal signal and the reconstruction will ideally be visible and it ispossible to interpret the anomalies.

In an embodiment, visualizing all individual files to detect theanomalies may prove to be unproductive, as it requires a substantialamount of time analyzing different files. To even further improve in thedetection of anomalies, an anomaly heatmap 1300 may be created in whichthe differences of the reconstructions and the original vibrations arevisualized. First color data points 1310-1320 correspond to thestrongest anomaly and second color data points 1340 represent the normaldata. At the x-axis of the heatmap in FIG. 13 the orders are visualizedfor the corresponding files while in the y-axis the timestampinformation (e.g. dates) of each sample file are defined.

For each original vibration yi belonging to Ck we have followingequation:C _(k)=max(y _(i) −ŷ _(i)).

where {circumflex over ( )}yi is the predicted value for original targetyi and k is each cell of the heatmap. Each cell serves as the spacingbetween the order grid in FIG. 13. It is a configurable parameter to aidin easier and more detailed analysis.

In an embodiment, anomaly heatmap information 1300 shown in FIG. 13 isdetermined for 30 files. On the x-axis the orders are arranged and therespective timestamp information (e.g. dates) of the files are arrangedat the y-axis. Color scale on the cells may be used to indicate theamount of difference. For example, a stronger color, such as in somecells of the area 1310, indicates a significant difference of the modelprediction and original data point between timestamps t1-t5 (e.g. 5consecutive days). The color intensity can be associated with apossibility for anomaly.

In an embodiment, based on the heatmap information 1300 it can bedetermined that the files corresponding to five consecutive timestamps(e.g. dates) observed by five consecutive stronger colored cells 1310,1320 are in the orders of interest. Cells 1310 are located in orders 9and 11 between timestamps t1 and t5 and cells 1320 are located in order18 between timestamps t1 and t5. The model clearly indicates these datapoints as anomalous. A false positive 1330 is also visible at 25th orderfor the file of timestamp t25. During analysis though this can beignored as it shows up on an order of no interest and is does notcontinue through time.

In an embodiment, after investigating the indicated anomalies from theheatmap information 1300, it is possible to analyze closer the filesindicated as anomalous. Differences between original and reconstructiondata may be determined. It is possible, for example, to detect for afile corresponding to timestamp t5 that the expected ghost frequency at12th order is reconstructed in contrast with the anomalies at 8.6 10.4and 17.2 orders.

In an embodiment, another example might be for a file that correspondsto timestamp t1 on the heatmap information 1300. Expected peaks arereconstructed while simulated anomalies are not. It could be determinedthat high peaks at 8th and 24th order are reconstructed and the modelrecognizes them as normal.

In an embodiment, in the case of the orders which represent an anomalouspattern (BPO and BPI), the corresponding cells in heatmap information1300 are clearly highlighted with a darker colored cell. As explained,this highlight represents the difference between the original andpredicted values. This difference is proven to be high when unnaturalhigh amplitudes are present in the original data. Observing the heatmapinformation 1300, as the first step in the new anomaly detectionprocedure, it saves time since it is possible to skip a number of filesthat are confirmed as normal. This is a faster approach than visualizingeach individual file and searching for anomalous peaks.

FIG. 14 shows another 3D heatmap information 1400 according to anexample embodiment.

The anomalies of the five files between t1 and t5 are visible in 8th,10th, and 17th order. This is an easy conclusion to reach, because theanomalies 1410, 1420 are present continuously throughout time, at they-axis. This is the case when a component is broken, as the anomaliesre-appear at the corresponding orders. The file of timestamp t24 isdetected to also contain a number of anomalies 1430. Thereconstruction-original differences seem to be present throughout thewhole file of t25 however. This can be detected to be due to a lowquality file, like a sampling anomaly or poor reconstructions from themodel.

In some instances it was concluded that alight anomaly for the pole passfrequency of 8^(th) order was present. This can lead to a false positiveanomaly, if we not investigated closer. This is due to the fact that thepole pass frequency (which is normal) and BPO fault frequency are bothin [8-9] order cell.

In an embodiment, for the above reason a complementary heatmapinformation may be determined with a higher resolution, of 0.1 order,for each cell. This way it can be clearly observed for which ordersthere are indicated anomalies, to avoid false positives.

FIG. 15 shows second 3D heatmap information 1500 according to an exampleembodiment.

The same sub-dataset of a plurality of files (e.g. 30) is used, but thistime a heatmap information 1500 of higher resolution is used, for orders8-18. It is clearly detectable to observe the BPI and BPO anomalies1510, 1520 exist for the first five files ranging from t1 to t5. Orders8 and 16, representing the pole pass, are also highlighted in cells1530, 1540 for the file of t24. By using a higher resolution, falsepositive anomalies, such as the pole pass frequency of 8th order, aredismissed. Peaks in the pole pass frequency are not considered anomalousand can therefore be ignored during the new analysis.

There are anomalies strongly present in the second heatmap information1500 and that can be confirmed by the spectra file as well.

In an embodiment, a procedure followed for this sub-dataset, representsa clear example of how this approach can be used. First, it isinvestigated the first heatmap information as in FIGS. 13-14. Afterdetecting anomaly indications, in the orders of interest, it is possibleto analyze closer with the higher resolution heatmap as in FIG. 15 toavoid false positives. Using the higher resolution heatmap information1500, it can be concluded that anomalies are present for specific faultfrequencies. Last, it is possible to define anomaly indicator dates tovisualize some of the spectra files as in FIG. 12, for example. Bydetermining the specific files of interest, it is confirmed that BPI andBPO anomalies exist for a thruster, for example.

In practice, it is important to define how much training data is neededto apply effective anomaly detection using the neural network 440 basedmodel 400, using an autoencoder model. From a practical aspect, usingseveral years of data might not be feasible. When it comes to thetraining data required by the model 400, to perform on a satisfactorylevel, it is pragmatic to determine the magnitude of informationrequired. If the anomalous pattern starts slowly appearing in thedataset then it is probable that the anomaly will be learned by themodel 400 and hence reproduced. This might occur for cases whenanomalies exist for several files, in the training dataset.

For this purpose, multiple experiments can be performed with differentnumber of spectra files used for training. It can be found, that it ispossible to perform anomaly detection according to different embodimentswith just 30% of the original data to achieve positive results. Tocompare experiments with different amount of training data in a justway, the number of epochs was increased for experiments with lesstraining data. Observing the loss curve, it is concluded that even 30%of the original data set suffices for training the neural network 440,with a sufficiently low validation loss score. This corresponds toapproximately one to two years of data in marine vessel environment.

In an embodiment, the model 400 may be used for other vibratingmachinery as well, not only thrusters. However, it is probable that theparameters of the neural network 440 will need to be modified for adifferent machine. The relation of the data points can be different thanthe patterns from thrusters.

Moreover, the reasoning behind the proposed solution is to build a modelthat learns the ordinary pattern of the vibrations, when no anomaliesare present. Unseen observations in meaningful orders are compared witholder information and thus a difference is observed in the validationdataset files. However, a normal pattern can change throughout thelifetime of equipment. The installed sensors can occasionally bereplaced or even moved. This might affect the normal pattern of thedata. It is therefore important that there is a process to handle suchdisturbance during analysis.

In an embodiment, it is worth pointing out that anomalies followspecific patterns. This effect, could be utilized also for supervisedlearning. If it is possible to collect a sufficient amount of specificanomalies, it is possible to deploy a model 400 for learning the variouspatterns and detect the anomalies by applying pattern recognitiontechniques. To apply this approach, there needs to be an investigationof the data for assembling sufficient datasets, of the same anomaly.

In an embodiment, the neural network 440 (see e.g. FIGS. 4-5) is trainedby means of signals from the individual sensors 410 for determininginternal neural network parameters.

FIG. 16 shows a schematic diagram of a sensor data item 124 inaccordance with an example embodiment. The sensor data item 124 asdisclosed may comprise at least one identifier.

In an embodiment, the propulsion system 150 and at least one sensordevice 290 (see e.g. FIG. 2) are configured to generate sensor dataitems based on the received identification system data and sensor data.The sensor data item may thus comprise sensor data generated by thesensor device 290 and an identifier information. The identifierinformation may comprise at least one of the following: sensor-ID(S-ID); propulsion-ID (P-ID), and vessel-ID (V-ID) that may comprise atleast part of the received automatic identification system (AIS) data,for example.

A sensor data item, such as propulsion vibration parameters 124, isgenerated by a sensor device of the marine vessel 105 (see FIG. 1) andtransmitted to control apparatus 120 and/or remote server apparatus 1730(see FIG. 17 or 18) as input data for the anomaly detection model (ADM)121, 400. Sensor data items may be processed at the control apparatus120 or server apparatus 1730 before inputting to ADM or transmitting tothe server apparatus or they may be inputted or sent without furtherprocessing.

FIG. 17 shows a schematic picture of a system 700 according to anexample embodiment. A marine vessel 105 may comprise a control apparatus120, for example. Instead of a marine vessel, the entity 105 maycomprise any setup utilizing an operating data system with a pluralityof sensors operationally connected thereto for measuring vibration data.

As an example, a marine vessel 105 is discussed. The marine vessel 105comprises the control apparatus 120 comprising means for generating,processing and transceiving vibration related data through acommunication interface, for example. The apparatus 120 is capable ofdownloading and locally executing software program code. The softwareprogram code may be a client application of a service whose possibleserver application is running on a server apparatus 1730, 731 of thesystem 1700. The marine vessel 105 comprises a plurality of sensorsoperationally connected to a vibration source to provide sensor data124. The apparatus 120 is configured to receive the sensor data 124 andprocess the data utilizing neural network and anomality detection model(ADM) 121, 400 (see FIG. 4). The ADM 121, 400 may be operated at aserver apparatus 1730 or at local apparatus 120. The apparatus 120 mayfurther comprise a capturing device, such a sensor device, for providingoperational or environmental data relating to the marine vessel 105 forproviding such data as further input to the ADM 121, 400 and neuralnetwork, for example. The sensor device may comprise an accelerometer ora gyroscope, for example. There may be a plurality of sensors thatcomprises a first sensor measuring vibration of a propulsion shaft and asecond sensor measuring vibration of a propeller blade, for example.

In an embodiment, there is provided a computer implemented method in amarine vessel data system, the method comprising: receiving data from aplurality of sensors configured to measure vibration and operationallyarranged to the marine vessel to provide time-domain reference sensordata; maintaining the time-domain reference sensor data within a datastorage system; generating a Fast Fourier Transform (FFT) on thetime-domain reference sensor data to provide a plurality of referencespectra files in frequency-domain, wherein each reference spectra filecomprises at least condition information and timestamp informationassociated to collection of the time-domain reference sensor data;normalizing each reference spectra file by converting frequency to orderinformation using the condition information (RPM) to provide normalizedreference spectra files; and training a convolutional autoencoder typeof neural network using the normalized reference spectra files.

In an embodiment, at least one reference vessel 1770-1772 is configuredto generate reference profile data 1782 by determining reference vesselparameters 1775 of the reference vessel 1770-1772. At least onereference vessel 1770-1772 is configured to generate reference sensordata 1783 based on sensors operationally connected to the referencevessel 1770-1772. Reference vessel 1770-1772 related measurements, datacollection and transceiving may be carried out by a reference dataapparatus 1780. A reference anomaly detection model 1784 may begenerated based on the reference profile data 1782 and the sensor data1783. The profile data 1782 may comprise, for example, operational orenvironmental characteristics, service data, oil measurement data, sparepart data, service provider manual input data etc. of the referencevessels 1770-1772.

The reference anomaly detection model 1784 may be transmitted to aserver apparatus 1730, 1731 for storing and processing over connection1781. The reference profile data 1782 and/or the reference sensor data1783 may also be transmitted to a server apparatus 1730, 731 for storingand processing. The reference anomaly detection model 1784 may also begenerated at the server apparatus 1730, 1731.

In an embodiment, reference profile data 1782, engine sensor data 1783or the reference anomaly detection model 1784 may be configured to bereceived at a control apparatus 120 that may comprise a propulsionsystem operated in the marine vessel 105, for example. At the apparatus120, sensor data 124 are received of the marine vessel 105, andanomality detection may be performed by selecting optimal model betweenthe reference model 1784 and the model 121, 400 (see FIG. 4), as well asby selecting between the sensor data 124 and the reference profile data1782 with the reference sensor data 1783.

In an embodiment, local operational or environmental parameters may bedetermined at the marine vessel 105 to define which sensor data 124 isused for the ADM 121, 400. In case the circumstances of the marinevessel 105 are such that there is not reliable history data to be usedfor the ADM 121, 400 (see FIG. 4), reference sensor data 1783 may beused as history data for the ADM 121, 400. Same applies if theenvironmental or operational parameters change and better environmentaldata associated with sensor data can be received from the referencevessels 1770-1772. The reference profile data 1782 may be configured toassociate reference sensor data 1783 to the operational (load, speed,service time, operating hours) and environmental (temperature, pressure,wind, humidity, wave height) circumstances and to assist the selectionof the most appropriate reference data 1783 to be used for the ADM 121,400 (see FIG. 4) eventually.

In an embodiment, an ADM 400 (see e.g. FIG. 4) may be operated at remoteserver 1730 and configured to receive reference data 1782-1784 from aremote apparatus 1780 comprising a plurality of reference sensorsoperationally arranged to a reference marine vessel 1770-1772 to providereference sensor data, determine reference historical data based on thereference sensor data; and provide the reference historical data asinput to the neural network 440 (see FIG. 4). Reference data may bereceived from a plurality of remote apparatuses each comprising aplurality of reference sensors to provide reference sensor data. Thereference data may relate to different operational conditions of thereference vessel 1770-1772. The reference data may relate to operationaland environmental measurement data of the reference vessel. Furthermore,reference data may be maintained at a server apparatus 1730, and thereference historical data may be dynamically updated based on thereference sensor data, and the reference historical data may be providedas input to the neural network 440 (see FIG. 4).

In the present description, by vessel are meant any kinds of waterbornevessels, typically marine vessels. Most typically the vessel is a cargoship or large cruise vessel, but the present disclosure is alsoapplicable for yachts, for example.

The control apparatus 120 is configured to be connectable at leastoccasionally to a public network 1750, such as Internet, directly vialocal connection or via a wireless communication network 1740 over awireless connection 1722. The wireless connection 1722 may comprise amobile cellular network, a satellite network or a wireless local areanetwork (WLAN), for example. The wireless communication network 1740 maybe connected to a public data communication network 1750, for examplethe Internet, over a data connection 1741. The control apparatus 120 maybe configured to be connectable to the public data communication network1750, for example the Internet, directly over a data connection that maycomprise a fixed or wireless mobile broadband access. The wirelesscommunication network 1740 may be connected to a server apparatus 1730of the system 1700, over a data connection.

In an embodiment, the control apparatus 120 may set up local connectionswithin the marine vessel 105 with at least one capturing device, such asa sensor, and a computer device. The capturing device, such as a sensor,may be integrated to the control apparatus 120, to a propulsion systemor to the marine vessel 105, attached to the hull of the marine vessel105 and connected to the vessel control system or arranged as separatesensor device and connectable over separate connection.

The control apparatus 120 and its client application may be allowed tolog into a vessel or propulsion data service run on a server 1730, forexample.

Real-time interaction may be provided between the control apparatus 120and the server 1730 to collaborate for marine vessel data and ADM 121,400 (see FIG. 4), over a network 1750. Real-time interaction may also beprovided between the apparatus 120 and the remote user device 1760 tocollaborate for marine vessel or vibration data over a network 1750,1761.

A sensor data item, such as vibration parameters 124, is generated by asensor device of the marine vessel 105 and transmitted to the controlapparatus 120 and/or to the server 1730. Sensor data items may beprocessed at the apparatus 120 before transmitting or they may be sentwithout further processing. Sensor data may also be stored within thecontrol apparatus 120 before transmission over the network 1750. Thenagain, transmitted sensor data may be stored/and or processed at theserver apparats 1730 or at the remote user device 1760.

A capturing device (e.g. a sensor device) may capture and send sensordata as a real-time content or non-real time data to the serverapparatus 1730 or to the remote user device 1760 over a peer-to-peerconnection formed over network, for example.

The control apparatus 120 may be connected to a plurality of differentcapturing devices and instruments and the apparatus 120 may beconfigured to select which sensor device(s) is actively collaboratedwith.

The user of the control apparatus 120 or the remote user device 1760 mayneed to be logged in with user credentials to a chosen service of thenetwork server 1730.

In an embodiment, the system 1700 comprises a sensor device configuredto be comprised by or connectable to the control apparatus 120 over alocal connection. The local connection may comprise a wired connectionor a wireless connection. The wired connection may comprise UniversalSerial Bus (USB), High-Definition Multimedia Interface (HDMI), or RCAinterface, for example. The wireless connection may comprise acousticconnection, Bluetooth™, Radio Frequency Identification (RF-ID) orwireless local area network (WLAN), for example. Near fieldcommunication (NFC) may be used for sensor device identification betweenthe sensor device and the control apparatus 120, for example.

A sensor device may also be connected directly to the public network1750, such as Internet, via direct local connection or via a wirelesscellular network connection 1740, 1741.

In an embodiment, the system 1700 may comprise a server apparatus 1730,which comprises a storage device 1731 for storing service data, servicemetrics and subscriber information, over data connection 1751. Theservice data may comprise configuration data; account creation data;sensor data; sensor ID's; reference data items; anomaly detection modelrelated data; user input data; real-time collaboration data; referencevessel profile data; reference vessel parameters; predefined settings;and attribute data, for example.

In an embodiment, a proprietary application in the control apparatus 120may be a client application of a service whose server application isrunning on the server apparatus 1730 of the system 1700.

In an embodiment, configuration information or application downloadinformation for any apparatus may be automatically downloaded andconfigured by the server 1730. Thus, the user of the devices may notneed to do any initialization or configuration for the service. Thesystem server 1730 may also take care of account creation process forthe service, sensor devices, apparatuses and users.

In an embodiment, the association of the devices can be one-time orstored persistently on any of the devices or the server 1730.

In an embodiment, authentication of a sensor device or control apparatus120 on a system server 1730 may utilize hardware or SIM credentials,such as International Mobile Equipment Identity (IMEI) or InternationalMobile Subscriber Identity (IMSI). The sensor device or controlapparatus 120 may transmit authentication information comprising IMEIand/or IMSI, for example, to the system server 1730. The system server1730 authenticates the device or control apparatus 120 by comparing thereceived authentication information to authentication information ofregistered users/devices/vessels/apparatuses stored at the system serverdatabase 1731, for example. Such authentication information may be usedfor pairing the devices and/or apparatuses to generate associationbetween them for a vessel or power plant data connection.

In an embodiment, a service web application may be used forconfiguration of a system. The service web application may be run on anyuser device, admin device, or a remote control device 1760, such as apersonal computer connected to a public data network, such as Internet1750, for example. The remote control apparatus 1760 may also beconnected locally to the control apparatus 120 over a local connection1723 and may utilize the network connections of the apparatus 120 forconfiguration purposes. The service web application of the remoteapparatus 1760 may provide searching/adding instruments, determiningattributes, device setup and configuration, for example. The service webapplication of the remote apparatus 1760 may be a general configurationtool for tasks being too complex to be performed on the user interfaceof the control apparatus 120, for example.

In an embodiment, a remote control apparatus 1760 may be authenticatedand configuration data sent from the control apparatus 1760 to thesystem server 1730, 1731, wherein configuration settings may be modifiedbased on the received data. In an embodiment, the modified settings maythen be sent to the control apparatus 120 over the network 1750 and thelocal connection or the wireless operator. The modified settings mayalso be sent to external devices correspondingly, through the controlapparatus 120 or directly over the network 1750, for example.

In an embodiment, the sensor device may be wireless or wired.

The system 1700 may also comprise a plurality of satellites 1710 inorbit about the Earth. The orbit of each satellite 1710 is notnecessarily synchronous with the orbits of other satellites and, infact, is likely asynchronous. A global positioning system receiverapparatus such as the ones described in connection with preferredembodiments of the present invention is shown receiving spread spectrumglobal positioning system (GPS) satellite signals 1712 from the varioussatellites 1710. The plurality of satellites 1710 may be used forlocation purposes, input for determining traveled distance (since oilchange or part change, for example), or input for accurate time (sinceoil/part change, for example).

The remote control apparatus 1760 may be configured to be operated by aremote operator of the vessel 105. The remote control apparatus 1760 maybe arranged on a ground station, on the vessel 105 or on another vessel,for example.

FIG. 18 presents an example block diagram of a server apparatus 1730 inwhich various embodiments of the invention may be applied.

The general structure of the server apparatus 1730 comprises a processor1810, and a memory 1820 coupled to the processor 1810. The serverapparatus 1730 further comprises software 1830 stored in the memory 1820and operable to be loaded into and executed in the processor 1810. Thesoftware 1830 may comprise one or more software modules, such as serviceapplication 1831 and can be in the form of a computer program product.The software 1830 may comprise the anomaly detection model (ADM) 121,400 and the service application 1831 may be configured to communicatewith the client application arranged at control apparatus 120, forexample. The client application may be configured to provide the sensordata from the marine vessel data system to the ADM model 400, 1830, forexample.

The processor 1810 may be, e.g., a central processing unit (CPU), amicroprocessor, a digital signal processor (DSP), a graphics processingunit, or the like. FIG. 18 shows one processor 1810, but the serverapparatus 1730 may comprise a plurality of processors.

The memory 1820 may be for example a non-volatile or a volatile memory,such as a read-only memory (ROM), a programmable read-only memory(PROM), erasable programmable read-only memory (EPROM), a random-accessmemory (RAM), a flash memory, a data disk, an optical storage, amagnetic storage, a smart card, or the like. The server apparatus 1730may comprise a plurality of memories. The memory 1820 may be constructedas a part of the server apparatus 1730 or it may be inserted into aslot, port, or the like of the server apparatus 1730 by a user. Thememory 1820 may serve the sole purpose of storing data, or it may beconstructed as a part of an apparatus serving other purposes, such asprocessing data.

The communication interface module 1850 implements at least part ofradio transmission. The communication interface module 1850 maycomprise, e.g., a wireless or a wired interface module. The wirelessinterface may comprise such as a WLAN, Bluetooth, infrared (IR), radiofrequency identification (RF ID), GSM/GPRS, CDMA, WCDMA, LTE (Long TermEvolution), or 5G radio module. The wired interface may comprise such asuniversal serial bus (USB) or National Marine Electronics Association(NMEA) 0183/2000 standard for example. The communication interfacemodule 1850 may be integrated into the server apparatus 1730, or into anadapter, card or the like that may be inserted into a suitable slot orport of the server apparatus 1730. The communication interface module1850 may support one radio interface technology or a plurality oftechnologies. Captured data associated with environmental data of themarine vessel 105, as well as measured parameters relating to operationconditions of the vessel may be received by the server apparatus 1730using the communication interface 1850.

The e-mail server process 1860, which receives e-mail messages sent fromcontrol apparatuses 120, such as marine vessel, and remote computerapparatuses 1760 via the network 1750. The server 1860 may comprise acontent analyzer module 1861, which checks if the content of thereceived message meets the criteria that are set for new activity dataitem of the service. The content analyzer module 1861 may for examplecheck whether the e-mail message contains a valid activity data item tobe used as reference data item. The valid reference data item receivedby the e-mail server is then sent to an application server 1840, whichprovides application services e.g. relating to the user accounts storedin a user database 1870 and content of the content management service.Content provided by the service system 1700 is stored in a contentdatabase 1880.

A skilled person appreciates that in addition to the elements shown inFIG. 18, the server apparatus 1730 may comprise other elements, such asmicrophones, displays, as well as additional circuitry such asinput/output (I/O) circuitry, memory chips, application-specificintegrated circuits (ASIC), processing circuitry for specific purposessuch as source coding/decoding circuitry, channel coding/decodingcircuitry, ciphering/deciphering circuitry, and the like. Not allelements disclosed in FIG. 18 are mandatory for all embodiments.

According to an embodiment, the server apparatus 1730 may receiveselection information for a plurality of reference vessels and generatethe reference profile data by determining reference parameters of theplurality of reference vessels based on the selection information. Thereference parameters may be generated based on sensor data received fromat least one reference vessel. The reference parameters may relate todifferent operation conditions of the reference vessel.

In an embodiment, a reference engine data apparatus 1780 (see FIG. 17)may be configured to determine the reference vessel sensor data,reference environmental data, and reference operational data, andgenerate reference model based on the reference inputs.

The reference model data of the reference model 1784 may be maintainedat a server apparatus 1730, and dynamically updated in response toreceiving updated reference vessel data.

The remote computer apparatus 1760 may comprise similar structure as thecontrol apparatus 120 of FIG. 2 but optionally without the sensordevices 260, 290 and GPS.

FIG. 19 shows a flow diagram showing operations in accordance with anexample embodiment of the invention. In step 1900, the method in amarine vessel data system is started.

In step 1910, time-domain data is received from a sensor configured tomeasure vibration of a system comprising a plurality of operationallyconnected parts, and operationally arranged to the marine vessel. Instep 1920, a Fast Fourier Transform (FFT) is generated on thetime-domain data to provide a plurality of spectra files infrequency-domain, wherein each spectra file comprises spectra datadefined by amplitude information and frequency information, and eachspectra file is associated with condition information determined basedon collection of the time-domain data. In step 1930, each spectra fileis normalized by converting the frequency information to orderinformation using the condition information to provide normalizedspectra files. In step 1940, actual sensor data is generated based onthe normalized spectra files. In step 1950, predicted data is generatedfor the at least one sensor as output by a convolutional autoencodertype of neural network, wherein the convolutional autoencoder type ofneural network is configured to be trained using normalized referencespectra files, wherein each normalized reference spectra file comprisesspectra data defined by amplitude information and order information, andeach spectra file is associated with timestamp information determinedbased on collection of the reference sensor data. In step 1960, thepredicted data is combined with the actual sensor data for the sensor toprovide error data. In step 1970, 3D anomaly heatmap information isgenerated, wherein a first dimension is defined by the timestampinformation, a second dimension is defined by the order information anda third dimension is defined by anomaly information determined based onthe error data. The method is ended in step 1980.

Without in any way limiting the scope, interpretation, or application ofthe claims appearing below, a technical effect of one or more of theexample embodiments disclosed herein is an improved system for a controlapparatus or an engine apparatus.

A technical effect of one or more of the example embodiments disclosedherein is that marine vessel performance is improved. A technical effectof one or more of the example embodiments disclosed herein is thatefficiency of a vessel is improved. A further technical effect of one ormore of the example embodiments disclosed herein is that operationalefficiency of a propulsion system, is improved. Another technical effectof one or more of the example embodiments disclosed herein is thatpossible faults or malfunctions are detected early enough to avoid anymajor break of marine vessel elements. Still another technical effect ofone or more of the example embodiments disclosed herein is that amountof false anomalies is minimized.

Although various aspects of the invention are set out in the independentclaims, other aspects of the invention comprise other combinations offeatures from the described embodiments and/or the dependent claims withthe features of the independent claims, and not solely the combinationsexplicitly set out in the claims.

It is also noted herein that while the foregoing describes exampleembodiments of the invention, these descriptions should not be viewed ina limiting sense. Rather, there are several variations and modificationsthat may be made without departing from the scope of the presentinvention as defined in the appended claims.

The invention claimed is:
 1. A computer implemented method in a marinevessel data system, the method comprising: receiving time-domain datafrom a sensor configured to measure vibration of a system comprising aplurality of operationally connected parts, and operationally arrangedto the marine vessel; generating a Fast Fourier Transform (FFT) on thetime-domain data to provide a plurality of spectra files infrequency-domain, wherein each spectra file comprises spectra datadefined by amplitude information and frequency information, and eachspectra file is associated with condition information determined basedon collection of the time-domain data; normalizing each spectra file byconverting the frequency information to order information using thecondition information to provide normalized spectra files; generatingactual sensor data based on the normalized spectra files; generatingpredicted data for the at least one sensor as output by a convolutionalautoencoder type of neural network, wherein the convolutionalautoencoder type of neural network is configured to be trained usingnormalized reference spectra files, wherein each normalized referencespectra file comprises spectra data defined by amplitude information andorder information, and each spectra file is associated with timestampinformation determined based on collection of the reference sensor data;combining the predicted data with the actual sensor data for the sensorto provide error data; and generating 3D anomaly heatmap information,wherein a first dimension is defined by the timestamp information, asecond dimension is defined by the order information and a thirddimension is defined by anomaly information determined based on theerror data.
 2. The method of claim 1, further comprising: detectingconsecutive spectra files of the actual sensor data in the firstdimension with anomaly information in the third dimension, wherein theanomaly information exceeds pre-defined anomaly threshold; determiningat least one detected spectra file to identify order information for theanomaly information; comparing the identified order information withpart frequencies of the plurality of operationally connected parts; anddetermining a faulty part based on the comparison.
 3. The method ofclaim 2, further comprising: identifying order information with at leastone harmonic for the anomaly information when determining the at leastone detected spectra file; comparing the identified order informationwith the at least one harmonic and the part frequencies of the pluralityof operationally connected parts; and determining the faulty part basedon the comparison.
 4. The method of claim 2, further comprising:detecting anomalies in a sub-range of the second dimension based on the3D anomaly heatmap information; and generating second 3D anomaly heatmapinformation, wherein a first dimension is defined by the timestampinformation, a second dimension is defined by the order information ofthe sub-range and a third dimension is defined by anomaly informationdetermined based on the error data.
 5. The method of claim 4, furthercomprising: determining false anomalies based on the second 3D anomalyheatmap information by identifying order pole pass frequencies.
 6. Themethod of claim 4, further comprising: detecting at least one spectrafile of the actual sensor data with anomaly information in the thirddimension, wherein the anomaly information exceeds pre-defined anomalythreshold; determining order information for the anomaly information inthe sub-range; comparing the determined order information with orderpole pass frequencies and part frequencies of the plurality ofoperationally connected parts; and determining the faulty part based onthe comparison.
 7. The method of claim 6, further comprising: ignoringorder information relating to pole pass frequencies when determiningfaulty part.
 8. The method of claim 1, wherein the at least one sensorcomprises an accelerometer.
 9. The method of claim 1, wherein thereceived time-domain data is maintained in a data storage system. 10.The method of claim 1, wherein generating the predicted data comprisesreconstructing data of the at least one sensor, by the neural network.11. The method of claim 1, wherein generating the predicted datacomprises determining correlation, by the neural network, for the atleast one sensor.
 12. The method of claim 1, wherein the neural networkis trained by means of signals from at least one sensor for determininginternal neural network parameters.
 13. The method of claim 1, whereinthe neural network is used for determination of the anomaly based on theerror data.
 14. The method of claim 2, wherein the sensor is configuredto measure vibration of a propulsion system comprising at least a shaft,a bearing, a propeller base and a propeller blade, and at least one ofthem having pre-defined part frequency provided for determining thefaulty part.
 15. The method of claim 1, wherein the training isconfigured to utilize a training function that comprises at least one ofthe following: Gradient descent function; Newton's method function;Conjugate gradient function; Quasi-Newton method function; andLevenberg-Marquardt function.
 16. The method of claim 1, furthercomprising: receiving remote normalized reference spectra filesoriginating from a remote marine vessel apparatus; and training theconvolutional autoencoder type of neural network using the remotenormalized reference spectra files.
 17. The method of claim 1, furthercomprising: receiving time-domain data from a plurality of sensorsconfigured to measure vibration of a system comprising a plurality ofoperationally connected parts, and operationally arranged to the marinevessel.
 18. A server apparatus in a marine vessel data system,comprising: a communication interface; at least one processor; and atleast one memory including computer program code; the at least onememory and the computer program code configured to, with the at leastone processor, cause the apparatus to: receive time-domain data from asensor configured to measure vibration of a system comprising aplurality of operationally connected parts, and operationally arrangedto the marine vessel; generate a Fast Fourier Transform (FFT) on thetime-domain data to provide a plurality of spectra files infrequency-domain, wherein each spectra file comprises spectra datadefined by amplitude information and frequency information, and eachspectra file is associated with condition information determined basedon collection of the time-domain data; normalize each spectra file byconverting the frequency information to order information using thecondition information to provide normalized spectra files; generateactual sensor data based on the normalized spectra files; generatepredicted data for the at least one sensor as output by a convolutionalautoencoder type of neural network, wherein the convolutionalautoencoder type of neural network is configured to be trained usingnormalized reference spectra files, wherein each normalized referencespectra file comprises spectra data defined by amplitude information andorder information, and each spectra file is associated with timestampinformation determined based on collection of the reference sensor data;combine the predicted data with the actual sensor data for the sensor toprovide error data; and generate 3D anomaly heatmap information, whereina first dimension is defined by the timestamp information, a seconddimension is defined by the order information and a third dimension isdefined by anomaly information determined based on the error data.
 19. Anon-transitory computer readable medium comprising computer executableprogram code, which code, when executed by at least one processor of anapparatus, causes the apparatus to: receive time-domain data from asensor configured to measure vibration of a system comprising aplurality of operationally connected parts, and operationally arrangedto the marine vessel; generate a Fast Fourier Transform (FFT) on thetime-domain data to provide a plurality of spectra files infrequency-domain, wherein each spectra file comprises spectra datadefined by amplitude information and frequency information, and eachspectra file is associated with condition information determined basedon collection of the time-domain data; normalize each spectra file byconverting the frequency information to order information using thecondition information to provide normalized spectra files; generateactual sensor data based on the normalized spectra files; generatepredicted data for the at least one sensor as output by a convolutionalautoencoder type of neural network, wherein the convolutionalautoencoder type of neural network is configured to be trained usingnormalized reference spectra files, wherein each normalized referencespectra file comprises spectra data defined by amplitude information andorder information, and each spectra file is associated with timestampinformation determined based on collection of the reference sensor data;combine the predicted data with the actual sensor data for the sensor toprovide error data; and generate 3D anomaly heatmap information, whereina first dimension is defined by the timestamp information, a seconddimension is defined by the order information and a third dimension isdefined by anomaly information determined based on the error data.